{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 机器学习练习 2 - 逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个笔记包含了以Python为编程语言的Coursera上机器学习的第二次编程练习。请参考 [作业文件](ex2.pdf) 详细描述和方程。\n",
    "在这一次练习中，我们将要实现逻辑回归并且应用到一个分类任务。我们还将通过将正则化加入训练算法，来提高算法的鲁棒性，并用更复杂的情形来测试它。\n",
    "\n",
    "代码修改并注释：黄海广，haiguang2000@qq.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练的初始阶段，我们将要构建一个逻辑回归模型来预测，某个学生是否被大学录取。设想你是大学相关部分的管理者，想通过申请学生两次测试的评分，来决定他们是否被录取。现在你拥有之前申请学生的可以用于训练逻辑回归的训练样本集。对于每一个训练样本，你有他们两次测试的评分和最后是被录取的结果。为了完成这个预测任务，我们准备构建一个可以基于两次测试评分来评估录取可能性的分类模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们从检查数据开始。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Exam 1</th>\n",
       "      <th>Exam 2</th>\n",
       "      <th>Admitted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34.623660</td>\n",
       "      <td>78.024693</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>30.286711</td>\n",
       "      <td>43.894998</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>35.847409</td>\n",
       "      <td>72.902198</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60.182599</td>\n",
       "      <td>86.308552</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>79.032736</td>\n",
       "      <td>75.344376</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Exam 1     Exam 2  Admitted\n",
       "0  34.623660  78.024693         0\n",
       "1  30.286711  43.894998         0\n",
       "2  35.847409  72.902198         0\n",
       "3  60.182599  86.308552         1\n",
       "4  79.032736  75.344376         1"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = 'ex2data1.txt'\n",
    "data = pd.read_csv(path, header=None, names=['Exam 1', 'Exam 2', 'Admitted'])\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们创建两个分数的散点图，并使用颜色编码来可视化，如果样本是正的（被接纳）或负的（未被接纳）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtcAAAHjCAYAAADojTN7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3X2U3GV9///XewkQ2d2KYMoPQtNQEm8QIdoVRVcCRJBGxSVSFuWnYKOx/aLkRmvi6TnenVYQ/TUu2q+WhiK2GJemm8VTU6siVoOVdqNRkGA32oAhUZAbnV2lQvb9++OaT3Z2d2Z3dvYz87l7Ps7JmZ3PzO5e+ezs7GuueV/vy9xdAAAAAOauLekBAAAAAHlBuAYAAABiQrgGAAAAYkK4BgAAAGJCuAYAAABiQrgGAAAAYkK4BgAAAGJCuAYAAABiQrgGAAAAYjIv6QHMxbOf/WxfvHhx0sMAAABAzu3atesX7r5gpvtlOlwvXrxYQ0NDSQ8DAAAAOWdmD9RzP8pCAAAAgJgQrgEAAICYEK4BAACAmGS65hoAACCrnnrqKe3fv19PPvlk0kNBhfnz5+vkk0/WkUce2dDnE64BAAASsH//fnV2dmrx4sUys6SHA0nurkcffVT79+/XKaec0tDXoCwEAAAgAU8++aSOP/54gnWKmJmOP/74Ob2bQLgGAABICME6feb6MyFcAwAAADEhXAMAAGRAqSRt2SJt3BguS6V4vu7g4KDMTPfff3/V26+66ipt27at7q934MABXXrppZKk3bt3a8eOHYdv+8Y3vqFvf/vbsx7j4sWL9Ytf/GLWn5cEwjUAAEDK7dwpLVworVsnXX99uFy4MByfq61bt6q7u1tbt26d+xeTdNJJJx0O43GF6ywhXAMAAKRYqSStXBkuR0fDsdHR8eMjI41/7ZGREe3cuVM33XSTvvCFL0gKHTPe+c536rnPfa5e9apX6eGHHz58/8WLF+t973ufli1bpq6uLn33u9/Vq1/9ap166qn6zGc+I0nat2+fTj/9dP32t7/V+9//fvX392vZsmX66Ec/qs985jPavHmzli1bpm9961t65JFH9IY3vEEveclL9JKXvER33XWXJOnRRx/VhRdeqBe84AV629veJndv/D/ZYk1rxWdmfy/ptZIedvfTy8eOk9QvabGkfZIuc/fHLVSO90laKenXkq5y9+82a2wAAABZ0d8vjY1Vv21sLNy+enVjX/v222/XRRddpOc85zk6/vjjtWvXLj3wwAP60Y9+pPvuu08///nPddppp+lP/uRPDn/OokWLtHv3bq1fv15XXXWV7rrrLj355JM6/fTT9ad/+qeH73fUUUfpwx/+sIaGhvSpT31KkvSb3/xGHR0des973iNJetOb3qT169eru7tbDz74oF796ldrz549+tCHPqTu7m69//3v15e+9CXddNNNjf0HE9DMPteflfQpSZ+rOLZJ0h3ufp2ZbSpf3yjpjyQtLf97qaRPly8BAAAKbXh4fMZ6stFRae/exr/21q1btXbtWknS5Zdfrq1bt+rpp5/WG9/4Rh1xxBE66aSTdP7550/4nIsvvliS9MIXvlAjIyPq7OxUZ2enjj76aD3xxBOz+v5f+9rXdN999x2+/qtf/UojIyP65je/qYGBAUnSa17zGj3rWc9q/D/ZYk0L1+7+TTNbPOnw6yWdW/74FknfUAjXr5f0OQ9z/t8xs2PN7ER3P9is8QEAAGTB0qVSe3v1gN3eLi1Z0tjXfeyxx/T1r39d99xzj8xMhw4dkpnpkksumfbzjj76aElSW1vb4Y+j608//fSsxjA2NqbvfOc7mj9//uz/AynV6prrEyoC888knVD+eKGkn1bcb3/52BRmtsbMhsxs6JFHHmneSAEAAFKgt1dqq5HY2trC7Y3Ytm2b3vzmN+uBBx7Qvn379NOf/lSnnHKKjj/+ePX39+vQoUM6ePCg7rzzzobH3tnZqVJFW5PJ1y+88EJ98pOfPHx99+7dkqRzzjlHn//85yVJ//qv/6rHH3+84TG0WmILGsuz1LOuTnf3G929y927FixY0ISRAQAApEdnp7RjR7hsbw/H2tvHj3d0NPZ1t27dOmWW+g1veIMOHjyopUuX6rTTTtNb3vIWnX322Q2P/bzzztN9992nZcuWqb+/X6973eu0ffv2wwsab7jhBg0NDemMM87QaaeddnhR5Ac+8AF985vf1Ate8AINDAxo0aJFDY+h1ayZqy/LZSH/UrGg8UeSznX3g2Z2oqRvuPtzzexvyx9vnXy/6b5+V1eXDw0NNW38qeYuDQ5KPT1S5U5CtY4DAIBU2bNnj57//OfXff+RkbB4ce/eUArS29t4sMb0qv1szGyXu3fN9Lmtnrn+oqQryx9fKen2iuNvseBlkn5JvfUMBgelVauk9etDoJbC5fr14fjgYLLjAwAAseroCF1Brr02XBKs06mZrfi2KixefLaZ7Zf0AUnXSbrNzFZLekDSZeW771Bow7dXoRXfW5s1rtzo6ZHWrpX6+sL1zZtDsO7rC8d7epIdHwAAQAE1s1vIG2vctKLKfV3S1c0aSy6ZhUAthUAdhey1a8NxSkIAAABajh0as6wyYEcI1gAAAIkhXGdZVGNdqbIGGwAAAC1FuM6qKFhHNdZjY+M12ARsAACARBCus2pwcDxYR6UgmzePB2y6hQDImFJJ2rJF2rgxXFbsMwEUm7u0ffvUibNax2fBzPTud7/78PWPf/zj+uAHPzjt5wwODk7YsryaZcuW6fLLL695+759+3T66afPaqzvf//79bWvfU2S9IlPfEK//vWvD9/2kY98ZFZfS5I++9nP6p3vfOesP28mhOus6umRBgYm1lhHAXtgIFfdQviDC+Tfzp3SwoXSunXS9deHy4ULw3Gg8JrYfvfoo4/WwMCAfvGLX8xiONOH6z179ujQoUP61re+pdFqe7Y36MMf/rBe9apXSYonXDcL4TqrzKRLLpm6eLHW8YziDy6Qf6WStHJluIz+Do+Ojh8fGUl2fEDiKtvvRgE7pva78+bN05o1a7R5coMEhdnl888/X2eccYZWrFihBx98UN/+9rf1xS9+UX/+53+uZcuW6cc//vGUz9u6dave/OY368ILL9Ttt99++PiuXbt05pln6swzz9Tf/M3fHD7+2c9+Vj09Pbrgggu0ePFifepTn9Jf//Vf60UvepFe9rKX6bHHHpMkXXXVVdq2bZtuuOEGHThwQOedd57OO+88bdq0Sb/5zW+0bNkyXXHFFZKkf/zHf9RZZ52lZcuW6R3veIcOHTokSbr55pv1nOc8R2eddZbuuuuuhs/bdAjXSC3+4ALF0N8flo1UMzYWbgcKbXLpZ1vb1NLQObj66qt166236pe//OWE4+9617t05ZVX6gc/+IGuuOIKXXPNNXr5y1+uiy++WB/72Me0e/dunXrqqVO+Xn9/vy6//HK98Y1v1NatWw8ff+tb36pPfvKT+v73vz/lc+69914NDAzov/7rv/QXf/EXOuaYY/S9731PZ599tj73uc9NuO8111yjk046SXfeeafuvPNOXXfddXrGM56h3bt369Zbb9WePXvU39+vu+66S7t379YRRxyhW2+9VQcPHtQHPvAB3XXXXdq5c+eMpS2NIlwjtfiDCxTD8PD4C+jJRkfDVs9A4TWx/e7v/M7v6C1veYtuuOGGCcf/4z/+Q29605skSW9+85u1s463jYeGhvTsZz9bixYt0ooVK/S9731Pjz32mJ544gk98cQTOueccw5/vUrnnXeeOjs7tWDBAj3zmc/U6173OknSC1/4Qu3bt29W/5877rhDu3bt0kte8hItW7ZMd9xxh37yk5/o7rvv1rnnnqsFCxboqKOOUm9v76y+br0I10gt/uCiaIq6vmDpUqm9vfpt7e3SkiWtHQ+QSk1uv7tu3TrddNNNc66R3rp1q+6//34tXrxYp556qn71q1/pn//5n2f8vKOPPvrwx21tbYevt7W16emnn57VGNxdV155pXbv3q3du3frRz/60YyLNONEuEZq8QcXRVLk9QW9veFd7mra2sLtmFlRX5wVQgva7x533HG67LLLdNNNNx0+9vKXv1xf+MIXJEm33nqrXvnKV0qSOjs7VaryABsbG9Ntt92me+65R/v27dO+fft0++23a+vWrTr22GN17LHHHp79vvXWW+c03sljOPLII/XUU09JklasWKFt27bp4YcfliQ99thjeuCBB/TSl75U//7v/65HH31UTz31lP7pn/5pTmOohXCN1OIPLvKmVvgp+vqCzk5px45wGb2gbm8fP97Rkez4sqDIL84KoUXtd9/97ndP6BryyU9+UjfffLPOOOMM/cM//IP6+vokSZdffrk+9rGP6UUvetGEBY3f+ta3tHDhQp100kmHj51zzjm67777dPDgQd188826+uqrtWzZMvkcXxCsWbNGF110kc4777zD18844wxdccUVOu200/SXf/mXuvDCC3XGGWfoggsu0MGDB3XiiSfqgx/8oM4++2y94hWv0POf//w5jaEWm+t/LkldXV0+NDSU9DDQRDt3hnAxNhbCRnt7CNY7dkjd3UmPDqjfdI/l++8PYajau7Ht7eFv5+rVrR9zq42MhLUUe/eGd6Z6ewnW9SiVQpCuNlPd2SkdOMB5TKs9e/bUF/DcQ4Du6ZlYY13rOOas2s/GzHa5e9dMnzuvaaMCYtDdHf4w8AcXWVY5Mx2JgvTKldLb3sb6Ain8XrfiRUSpFJ5ThodD+VlvbwihWVXP4u8ivDjLtajNbr3HkSjCNVKvVX9wgWaZKfw8+miYoa41c836gvhUewdhw4ZsvxvG4m8gXai5BoAmmyn8HH886wtaIa+17Sz+zrYsl+fm1Vx/JoTrtHCXtm+fuuK31nEAmTFT+HnBC1jQ1wp57Z3P4u/smj9/vh599FECdoq4ux599FHNnz+/4a9BWUhaDA5Kq1ZNXAlc2XpnYIC6KiCjentD6UE1Ufjp6GB9QbPltXwiehFWa8Esj6H0Ovnkk7V//3498sgjSQ8FFebPn6+TTz654c8nXKdFT894Sx0pBOzKnpY9PcmOD0DD6g0/rC9orugdhDzWtrP4O5uOPPJInXLKKUkPAzGjFV+aVM5URypnsgFkGq3mkkXLOgBzUW8rPsJ12rhPLJ4bGyNYA0BM6J0PoFH0uc6iaOa60vr1zFwDQEwonwDQbITrtKgsCYlKQSpLRAjYABALatsBNBPhOi0GBycGa7NwKYXjy5fTLQQAACDlCNdp0dMT2u319IzPUEcBe/lyuoUAAABkAOE6Lcyqz0zXOg4AAIDUYYdGAAAAICaEawAAACAmhGsAAAAgJoRrAAAAICaEawAAACAmdAsBAGROqRR2WRwelpYuDbssdnYmPSoAIFwDADJm505p5UppbEwaHZXa26UNG6QdO8L25gCQJMpCAACZUSqFYF0qhWAthcvo+MhIsuMDAMI1ACAz+vvDjHU1Y2PhdgBIEuEaAJAZw8PjM9aTjY5Ke/e2djwAMBk11wCAzFi6NNRYVwvY7e3SkiWtH1PWsTgUiJe5e9JjaFhXV5cPDQ0lPQwAQIuUStLCheFyss5O6cABqaOj9ePKqmqLQ9vaWBwKVGNmu9y9a6b7URYCAMiMzs4Q/Do7QxCUwmV0nGBdPxaHAs1BWQgAIFO6u8MMdX9/qLFesiSUMhCsZ6eexaGrV7d2TEAeEK4BAJnT0UHwmysWhwLNQVkIAAAFFC0OrYbFoUDjCNcAABRQb29YvFhNW1u4HcDsEa4BACigysWhxxwTjs2bJx19tLRtGzXsQKMI1wAAFFR3dwjSY2PSkUdKTz8dAvall4Y2fQBmj3ANAEBBlUohSD/5pPTUU+EY7fiAuSFcAwDQJKWStGWLtHFjuKy2+U2S6mnHB2B2aMUHAEATVNv9cMOGdO1+SDs+IH7MXAMAELOs7H5IOz4gfomEazNba2b3mtkPzWxd+dhxZvZVMxsuXz4ribEBADBXWSm3oB0fEL+Wh2szO13S2yWdJelMSa81syWSNkm6w92XSrqjfB0AgMzJSrlFZTu+aAa7vX38OO34gNlLoub6+ZLudvdfS5KZ/bukVZJeL+nc8n1ukfQNSRsTGB8AALNSKoXZ6OHhUGrxe78XQmq1gJ22covubunAgTD+vXvD2Hp7CdZAo5II1/dK+iszO17SbyStlDQk6QR3P1i+z88knVDtk81sjaQ1krRo0aLmjxYAgGlUW7hoVrssJI3lFh0d0urVSY8CyIeWh2t332NmH5X0FUmjknZLOjTpPm5mXuPzb5R0oyR1dXVVvQ+AdJs8y9fbG96GBrKmcuFiJJqtPuaYEFrdx0N3WxvlFkDeJdKKz91vknSTJJnZRyTtl/RzMzvR3Q+a2YmSHk5ibACaKwvtyYB6Tbdw0Uy67jpp/nzKLYAiSSRcm9nvuvvDZrZIod76ZZJOkXSlpOvKl7cnMTYAzTPdLN/KlaHuk+CBLJlp4eL+/dK117Z2TACSldQmMv9crrl+StLV7v6EmV0n6TYzWy3pAUmXJTQ2AE1ST3sy6j6RJVGf6CwsXES6US6XH0mVhbyyyrFHJa1IYDhAauT9yTUr7cmAevX2hrKmatK4cBHpRLlcvrD9OZASRXhyZZYPeRP1g578u8vCRdSLcrn8YftzIAWyslXyXLEbHPIo6hPd1ydt2hQuDxzIz4tiNFdWdvNE/Zi5BlKgKLXIzPIhr+gTjUZRLpc/hGsgBYr05MpucAAwjnK5/CFcAylQtCdXZvkAFMl0i9VZFJs/5p7dTQ67urp8aGgo6WEAc1YqSQsXTlzQEunsZEELkCd57wqEiaotVo9K4aK6/Hrug+SZ2S5375rxfoRrIB14cgXyj9/zYpnNxMnICOVyaVdvuKYsBLmWpRkiapGBfKPlWvHMZrE65XL5QbhGbmWxbzRPrkB+FaUrEMYVabE6xhGukUvMEAFIG4JW8RRtsXo1WXoHOS5sIoNcoik/gLSJglY1RQlaRVP0jbN27gw15+vWSddfHy4XLgzH84xwjVxihghJKJWkLVukjRvDZbVFTCiuogetIoo2zursHH9h1d4+fjzP76AWZefhaigLQS7xVhxaLYs1/mgtdigtpqIuVi/yGgPCNXKJpvxoJWr8Ua+iBq2iK+Ji9SK/g0y4Ri4xQ4RWKvIMDWqrtZCriEELxVPkd5AJ18gtZojQKkWeoZmNInUNoEwIRVfkd5AJ18g1ZojQCkWeoalXkcImZUJAsd9BplsIAMwRXSCmV7SuAbQCBYLoHeS+PmnTpnB54ED+XlBPxsw1AMxRkWdo6lG0mnTKhIBxRXwHmXANADGgxr+2ooVNyoSAYiNcA0BMijhDU4+8hs1aCzSLvJALgGTunvQYGtbV1eVDQ0NJDwMAMI1SKWx5XG3Hys7ObC7wq7ZAMyoD6u6e+XYA2WNmu9y9a6b7MXMNAGiqvNWk19MNhDIhoLgI1wCApstT2Kx3gSZlQkAxEa4BAC2Rl7BZtAWaAGaHPtcAAMxCtECzmiwv0AQQD8I1AACzwKZBAKZDuAYAYBaiBZqdneMz2O3t48ezWEcOID7UXAMAMEt5WqAJIF6EawAAGpCXBZoA4kVZCAAAABATwjUAAAAQE8I1AAAAEBPCNQAAABATwjUAAAAQE8I1AAAAEBPCNQAAABATwjUAAAAQE8I1AAAAEBPCNQAAABATwjUAAAAQE8I1AAAAEJN5SQ8AABC/Uknq75eGh6WlS6XeXqmzM+lRAUD+Ea4BIGd27pRWrpTGxqTRUam9XdqwQdqxQ+ruTnp0AJBvlIUAQI6USiFYl0ohWEvhMjo+MpLs+AAg7wjXAJAj/f1hxrqasbFwOwCgeQjXAJAjw8PjM9aTjY5Ke/e2djwAUDSEawDIkaVLQ411Ne3t0pIlrR0PABRNIuHazNab2Q/N7F4z22pm883sFDO728z2mlm/mR2VxNgAIMt6e6W2Gs/sbW3hdgBA87Q8XJvZQknXSOpy99MlHSHpckkflbTZ3ZdIelzS6laPDQCyrrMzdAXp7ByfwW5vHz/e0ZHs+AAg75JqxTdP0jPM7ClJx0g6KOl8SW8q336LpA9K+nQiowOADOvulg4cCIsX9+4NpSC9vQRrAGiFlodrd3/IzD4u6UFJv5H0FUm7JD3h7k+X77Zf0sJqn29mayStkaRFixY1f8AAkEEdHdJq3v8DgJZLoizkWZJeL+kUSSdJapd0Ub2f7+43unuXu3ctWLCgSaMEAAAAZi+JBY2vkvQ/7v6Iuz8laUDSKyQda2bRTPrJkh5KYGwAAABAw5II1w9KepmZHWNmJmmFpPsk3Snp0vJ9rpR0ewJjQx64S9u3h8t6jgMAAMSk5eHa3e+WtE3SdyXdUx7DjZI2StpgZnslHS/pplaPDTkxOCitWiWtXz8epN3D9VWrwu0AAABNkEi3EHf/gKQPTDr8E0lnJTAc5E1Pj7R2rdTXF65v3hyCdV9fON7Tk+z4ALRMqRS6pgwPhw12entDW0IAaBbzDL9F3tXV5UNDQ0kPA2kUzVRHAVsKwXrzZsksuXEBaJmdO6WVK6WxsbD1e3t72Ehnx47QrhAAZsPMdrl714z3I1znjHsoe+jpmRgiax3PM/eJW9WNjRXn/45CY7Y2nIOFC8PlZJ2doQ84fb8BzEa94TqR7c/RRNQbB9H/uVLlOQFyaufOECrXrZOuvz5cLlwYjhdJf394PV3N2Fi4HQCagXCdN5X1xlGYLFq98eT/89jY1HMC5FCpFMogSqVQBiGFy+j4yEiy42ul4eHxczDZ6GjYuRIAmiGp7c/RLGahrlgKYTKqOS5SvfHg4Hiwjv7Pledk+XLpkkuSHSPQBPXM1hZl18alS0ONdbWA3d4etoQHgGZg5jqPKsNkpCjBWgqz8wMDE//P0TkZGCjG7D0Kidnacb29E5dcVGprC7cDQDMQrvOo6PXGZmFmevKLiVrHgZyIZmurKdpsbWdn6ArS2Tl+Ttrbx4+zmBFFUSpJW7ZIGzeGy2qLfBEvuoXkzeR648k9nos0gw0UDB0yphoZCeUwe/eGFxe9vcU7Bygu2lHGi1Z8RbV9e+gKUhmkKwP3wAD1xkCO8ccUgMSL7WaoN1yzoDFvonrjyn7WUb3x8uXUGwM5190d/mgyWwsUGwuck0O4zpuorrje4wByp6ODP5poPTYvShcWOCeHcA0AAOakWjnShg2UIyWJdpTJoVsIAABoGJsXpRPtKJNDuEZy3MMCzMmLamsdBwCkDlvNpxPtKJNDWQiSMzhIZ5OEUSM5Pc4PMDNqe9OLBc7JIFwjOT09IVhHW7RP7slNZ5OmokZyepwfoD7U9qYbC5xbjz7XSFblTHWEzW6ajv6n0+P8APXj9wVFUW+fa2qusy7rdctRD+5KBOumo0Zyepwf5EGrtr2mtheYiHCddVHd8vr140E6mg1etSrcnmbRWCtV/l/QFNRITo/zg6zbuTPMJq9bJ11/fbhcuDAcb4aotrevT9q0KVweOEAJFYqJmuusy3LdcmVJSFQKUlkiwgx201AjOT3OD7KssjVeJHosr1zZvDINanuBgJnrrIvKKqKA3dY2MaymOZwODk4da+X/Je2z7hlG/9PpcX6QZZQ1AckiXOdBVuuWe3pCu73KsUb/l4GBdM+6Zxw1ktPj/CDLKGsCkkVZSB7UqltOe8A2q97HutZxxIr+p9Pj/CCrKGsCkkUrvqybrm45C6UhAIBY0RoPaI56W/Exc511teqWpXB8+XJmgQGgQKLypcmbILW1TSxrYgdSoDmYuc469xCwe3omzlDXOg4AKISRkdplTdV2II3CN+3zgOrqnbkmXAMAUCCUjQCNYYdGAAAwBa36gOYiXAMAUCC06gOai3ANAECBRK36qqFVHzB3hGsAiFmpJG3ZIm3cGC6r1bYCSWEHUqC5CNdF5i5t3x4u6zkOYEY7d4bFYuvWSddfHy4XLgzHI4RvJIkdSIHmoltIkW3fLq1aNbFHduWmNAMD9MgGZqGeLgy7d9MCDekwXas+AFOxiQxm1tMTgnVfX7g+eXfHnp5kxwdkzExdGG65RXrf+yaG72hh2cqVtEBDa3V0SKtXJz0KIH8I10U2eTfHKGSzbTrQkJm6MPzLv8zcAo2wAwBTZWlHUWqui64yYEcI1kBDZurCINECDQBmq561LGlCuC66qMa60vr1LGYEGjBTF4bXvIYWaAAwG6VSKJsrlcYnJ0ZHx4+PjCQ7vmoI10VWuXhx7drwvnRUg03ABmZtpi4MV15JCzQAmI0s7ihKzXWRDQ6OB+uoFKSyBnv5crqFALPU3R0WJtbqwrBjR+1uISxmBICJsrijKOG6yHp6Qru9np7xGusoYC9fTrcQoEHTdWGYKXwDAMZFa1mqBey0ltPR5xoAAACpVM/+Aa2anKi3zzU110gfdo4EAADK5o6ihGukz+Bg2DmyclFltPhy1apwOwAAKISonK6vT9q0KVweOJDeXW2puUb6sHMkAACokKUdRQnXSB92jgQAABnFgkakl/vEpsBjYwRrAIWRpe2egSJgQSOyjZ0jARRY1rZ7BjCOcI30YedIAAWWxe2eAYxrec21mT1XUuVmlX8g6f2SPlc+vljSPkmXufvjrR4fUoCdI1FwlAOkW7N/PvVs95yVhV15w+8m6jFjzbWZPUfSpyWd4O6nm9kZki5297+c8zc3O0LSQ5JeKulqSY+5+3VmtknSs9x943SfT811TrmHgF25c+R0x4Ec2bmz9vboaW07VSSt+Pls3BhKQWrZtEm69tp4vhfqx+8m4qy5/jtJ75P0lCS5+w8kXT634R22QtKP3f0BSa+XdEv5+C2S6LdWVGZhZnpygK51HMgJygHSrVU/n2i752rSut1z3vG7idmoJ1wf4+7/OenY0zF9/8slbS1/fIK7Hyx//DNJJ1T7BDNbY2ZDZjb0yCOPxDQMAPUqlaQtW8Ls2pYt1bekRWPqKQdAclr18+ntndgoqVJbW7gdrcXvJmajnnD9CzM7VZJLkpldKung9J8yMzM7StLFkv5p8m0ealWq1qu4+43u3uXuXQsWLJjrMADMAh0MmmuLDljnAAAgAElEQVR4eHxWbLLRUWnv3taOBxO16ueTxe2e847fTcxGPQsar5Z0o6TnmdlDkv5H0hUxfO8/kvRdd/95+frPzexEdz9oZidKejiG7wEgJpVvi0aiPzYrV4ataPmjPzdROUC1P+KUAySvlT+faLvn/v4Q3JYsCTPWHR0sqksCv5uYjWkXNJpZm6RL3f02M2uX1ObusbwJbGZfkPRv7n5z+frHJD1asaDxOHd/73RfgwWNQOts2RJmqmv9cenro4PBXJVK4Z2AaqU2nZ28gElaGn4+LKpLRhp+9lk0lxeCaXwRWe+Cxmlnrt19zMzeK+k2d6/xhkhDg2uXdIGkd1Qcvk7SbWa2WtIDki6L6/sBmDveFm2+6G3/WuGJP97JSvrnw7tHyUn6Z59F1V4IbthQ3wvBuXxuGtRTFvI1M3uPQg/qw39a3f2xRr9pOagfP+nYowrdQwCkEG+LtsZ05QBIXpI/H/pfJ4vfzfrN5YVgHl5E1hOuo3XJV1ccc4XNXwAURG9vmDmohg4G8eroICSlWVI/H949Sh6/m/WZywvBPLyInDFcu/sprRgIgHTjbVEgWbx7hKyYywvBPLyInDFcm9mRkv5M0jnlQ9+Q9Lfu/lQTxwUghXhbFEgO7x4hK+byQjAPLyLr2f58i6QjNb574pslHXL3tzV5bDOiWwgAoEjoFoIsmEt3lTR3ZomlW0jZS9z9zIrrXzez7zc+NAAA0AjePUIWzKWMMA8liPWE60Nmdqq7/1iSzOwPJB1q7rAAAEA1LKpDFszlhWDWX0TWE67/XNKdZvYTSSbp9yW9tamjAgAAQKbN5YVgll9E1tMt5A4zWyrpueVDP3L3/23usAAAAIDsaZvpDmZ2taRnuPsP3P0Hko4xs//T/KEBAAAA2TJjuJb0dnd/Irri7o9LenvzhgQAAABkUz0110eYmXm5Z5+ZHSHpqOYOCwCQFaVSWHg0PBx61Pb2hhX/AFBE9cxcf1lSv5mtMLMVkraWjwHF4i5t3x4u6zkOFMDOnaEn7bp10vXXh8uFC8NxACiiesL1RklfV9il8c8k3SHpvc0cFJBKg4PSqlXS+vXjQdo9XF+1KtwOFEipFHrRlkrju6mNjo4fHxlJdnwAkIQZw7W7j7n7ZyS9SdJfSdru7vS5RvH09Ehr10p9feMBe/36cH3t2nA7UCD9/WGTh2rGxsLtAFA0NWuuzewzkj7p7j80s2dK+g+FzWOOM7P3uPvWVg0SSAUzafPm8HFfX/gnhWC9eXO4HSiQ4eHxGevJRkfD5g9AUbD2AJHpZq5f6e4/LH/8Vkn/7e4vlPSHoiwERVUZsCMEaxTU0qVhW+Jq2tvDrmpAEbD2AJWmC9e/rfj4AkmDkuTuP2vqiIA0i0pBKlXWYAMF0tsrtdX4K9LWFm4H8i5Law9KJWnLFmnjxnBZKiU9onyaLlw/YWavNbMXSXqFyh1CzGyepGe0YnBAqkyusR4bm1qDDRRIZ6e0Y0e4jGaw29vHj3d0JDs+oBWysvaA2fXWma7P9Tsk3SDp/5G0rmLGeoWkLzV7YEDqDA6OB+uoFKSyBnv5cumSS5IdI9Bi3d3SgQMhQOzdG0pBensJ1iiOLKw9qJxdj0RjXrky/A7zOxufmuHa3f9b0kVVjv+bpH9r5qCQI+4hlPb0TKxLrnU8zXp6pIGBiWOOAvby5XQLaSIWCqVbR4e0enXSowCSEa09qBaw07L2oJ7ZdX6H41NPn2ugcXnqDW0WZqYnvxiodRyx4K1MAGmWhbUHWZhdzxPCNZqL3tCYgywtFAJQTFlYe0Bnn9aaruYamDt6Q2MOeCsTQBakfe1Bb6+0YUP129Iyu54n04ZrM3uepIWS7nb3kYrjF7n7l5s9OOREFLCjYC0RrFEX3soEkBVpXnsQzaKvXBkmJkZHw4x1W1t6ZtfzpGZZiJldI+l2Se+SdK+Zvb7i5o80e2DIEXpDo0G8lQkA8Yhm1/v6pE2bwuWBA+E44jXdzPXbJf2hu4+Y2WJJ28xssbv3SWLKEfWZXGO9efP4dYkZbEyLtzIBID5pnl3Pk+nCdVtUCuLu+8zsXIWA/fsiXKNe9IbGHPBWJgAga8xrvDVvZl+XtMHdd1ccmyfp7yVd4e5HtGaItXV1dfnQ0FDSw8B08tTnGokZGUnvQiEAQDGY2S5375rxftOE65MlPV2xM2Plba9w97vmPsy5IVwDAACgFeoN19Pt0Lh/mtsSD9YAAABA2rCJDID6uUvbt0/t9FLrOAAABUO4BlC/PG1nDwBAE9S9Q6OZ/U7l/d39saaMCEB6VW5nL01srch29gAAzByuzewdkj4k6UlJ0Xu+LukPmjguAGnEdvYAAEyrZreQw3cwG5Z0trv/ojVDqh/dQoCEuIdm05GxMYI1ACDX6u0WUk/N9Y8l/XruQwKQC2xnDwBATfXUXL9P0rfN7G5J/xsddPdrmjYqAOnEdvYAAEyrnnD9t5K+LukeSWPNHQ6AVGM7ewAAplVPuD7S3Tc0fSQA0q+nRxoYmLhtfRSwly+nWwgAoPDqqbn+VzNbY2Ynmtlx0b+mjwxA+piFmenJpR+1jgMAUDD1zFy/sXz5vopjtOIDAAAAJpkxXLv7Ka0YCAAAAJB1de3QaGanSzpN0vzomLt/rlmDAgCkU6kk9fdLw8PS0qVSb6/U2Zn0qADkSdafZ+rZROYDks5VCNc7JP2RpJ3ufmnTRzcDNpEBEuIeOodULmyc7jhyYedOaeXKsGfQ6KjU3h72EtqxQ+ruTnp0APIgzc8zcW4ic6mkFZJ+5u5vlXSmpGfOcXwAsmxwUFq1auLmMVEP7FWrwu3IlVIp/MErlcIfPClcRsdHRpIdH4Dsy8vzTD3h+jfuPibpaTP7HUkPS/q95g4LDXGXtm+fulNereNAo3p6Qq/rvr7xgF25uQwt+XKnvz/MJFUzNhZuB4C5yMvzTD3hesjMjpX0d5J2SfqupP9o6qjQGGYT0SpRb+soYLe1Td1cBrkyPDw+kzTZ6Ki0d29rxwMgf/LyPDNjuHb3/+PuT7j7ZyRdIOnKcnkI0obZRLRS5e6MEYJ1bi1dGmofq2lvl5Ysae14AORPXp5nZgzXZrY6+tjd90n6YXmRY8PM7Fgz22Zm95vZHjM7u7w5zVfNbLh8+ay5fI9CYjYRkVaUCEUv3ipVvmuCXOntDU8p1bS1hdsBYC7y8jxTT1nICjPbUd6h8QWSviNprg1R+iR92d2fp7BAco+kTZLucPelku4oX8dsMZsIqfklQpPfFRkbm/quCXKlszOs1u/sHJ9Zam8fP97Rkez4AGRfXp5n6tlE5k1m1ivpHkmjkt7k7nc1+g3N7JmSzpF0Vfnr/1bSb83s9Qot/yTpFknfkLSx0e9TWLVmEwnYxVJZIiSFn3+cJUKDg1PfFYle1PX1ScuXh+3QUyLrPVPTortbOnAgnMu9e8NbtL292fmDByD98vA8U0+f66UKYfceSc+XdJ+kDe7+64a+odkySTeWv86ZCosk10p6yN2PLd/HJD0eXZ/0+WskrZGkRYsW/eEDDzzQyDDyafJs4uRARcAulsrHQySux0GG+lynuWcqACA76u1zXU+4vl/S1e5+Rzn0bpD0J+7+ggYH1qVQWvIKd7/bzPok/UrSuyrDtJk97u7T1l2zicwk27eHt/wrA1RlwBoYSNVsIlrAfWIB29hYakJvK5RK0sKF4XKyzs4wO5Kl2RAAQHLi3ETmLHe/Q5I8+P8kzSWh7Ze0393vLl/fJunFkn5uZidKUvny4Tl8j2Lq6QkBunJmMnq7fmCAbiFFw4LD3PRMBQBkR81wbWbvlSR3/5WZ/fGkm69q9Bu6+88k/dTMnls+tEKhROSLkq4sH7tS0u2Nfo/CMgsz05NnJmsdR36x4FBSfnqmAgCyY7qZ68srPn7fpNsumuP3fZekW83sB5KWSfqIpOskXWBmw5JeVb4OoBG1FhxGAbsgGwrlpWcqACA7atZcm9n33P1Fkz+udj0p1FwDNWRowWEzUXMNAIhLHDXXXuPjatcBpAklQpLy0zMVAJAd0/W5PtPMfiXJJD2j/LHK1+c3fWQAEIM89EwFAGRHzXDt7ke0ciAA0CwdHdLq1UmPAgBQBPW04gMAAABQB8I1AAAAEBPCNQAAABATwjUAAAAQk+m6hQAAAKROqRQ6AA0Ph82ientDi00gDQjXAAAgM3bulFaulMbGpNHR0Lt+w4bQu767O+nRAZSFAACAjCiVQrAulUKwlsJldHxkJNnxARLhGgAAZER/f5ixrmZsLNwOJI1wDTTKXdq+PVzWcxwAMEWpJG3ZIm3cGC5Lpdr3HR4en7GebHQ07MIKJI1wDTRqcFBatUpav348SLuH66tWhdsBADXt3CktXCitWyddf324XLgwHK9m6dJQY11Ne7u0ZEnzxgrUi3ANNKqnR1q7VurrGw/Y69eH62vXhtsBAFU1Uj/d2yu11UgubW3hdiBphGugUWbS5s3jAbutbTxYb94cbgcAVNVI/XRnZ+gK0tk5PoPd3j5+vKOjeeMF6kUrPmAuooDd1zd+jGANADNqtH66u1s6cCCE7717QylIby/BGulBuAbmIioFqbR+PQEbAGYQ1U9XC9gz1U93dEirVzdvbMBcUBYCNGpyjfXY2NQabABAVdRPI68I10CjBgen1lhX1mDTLQQAaqJ+GnllnuHZta6uLh8aGkp6GCgq9xCge3omloDUOg4AmGJkhPppZIOZ7XL3rhnvR7gGAAAApldvuKYsBAAAAIgJ4RoAAACICeEaAAAAiAnhGgAAAIgJ4RoAAACICeEaAAAAiAnhGgAAAIgJ4RoAAACICeEaAAAAiAnhGgAAAIgJ4RoAAACIybykBwAAKK5SServl4aHpaVLpd5eqbMz6VEBQOMI1wCAROzcKa1cKY2NSaOjUnu7tGGDtGOH1N2d9OgAoDGUhQAAWq5UCsG6VArBWgqX0fGRkWTHBwCNIlwDSJ67tH17uKznODKvvz/MWFczNhZuB4AsIlwDSN7goLRqlbR+/XiQdg/XV60KtxdBgV5kDA+Pz1hPNjoq7d3b2vEAQFwI10AeZD2U9fRIa9dKfX3jAXv9+nB97dpwexEU6EXG0qWhxrqa9nZpyZLWjgcA4kK4BvIg66HMTNq8eTxgt7WNB+vNm8PtRVCgFxm9veHHXE1bW7gdALLIPO0zWtPo6uryoaGhpIcBJG9yCNu8eer1LARU94mJa2wsG+OOU+XPMpKln+EsVOsW0tZGtxAA6WRmu9y9a8b7Ea6BnMh6KMv6+ONUoBcZIyNh8eLevaEUpLdX6uhIelQAMFW94ZqyECAvotKKSlkJppNn3sfGppZHFEV0Lirl+Bx0dEirV0vXXhsuCdYAso5wDeRFlkPZ4ODUEpbKGuy014zHhRcZAJB5hGsgD7Ieynp6pIGBiTPtUcAeGMjVQr5p8SIDADKPmmsgD7ZvD11BKkNZZeAeGJAuuSTpUWIm7iFA9/RMLOepdRwA0DIsaASKhFAGAEBTsaARKBKzMDM9OUDXOo50yPrmPwCAKQjXAJCUrG/+AwCYYl4S39TM9kkqSTok6Wl37zKz4yT1S1osaZ+ky9z98STGBwAtUbkjozR185+iLOQEgBxJcub6PHdfVlG7sknSHe6+VNId5esA4kQZQrqw7TsA5E6aykJeL+mW8se3SErvlA0BBVlFGUL6ZHnzHwDAFEmFa5f0FTPbZWZrysdOcPeD5Y9/JumEZIZWBwIKsqqyDCF6/FKGkKwsb/4DAJgikZprSd3u/pCZ/a6kr5rZ/ZU3urubWdW/LOUwvkaSFi1a1PyRVkOdJLKqcpa0r2/8MUwZQjImv7ipfC6R+JkAQAYl3ufazD4oaUTS2yWd6+4HzexESd9w9+dO97mJ9rmu/KMYIaAgK9xDfW9kbIzHbRLY/AcAMiO1fa7NrN3MOqOPJV0o6V5JX5R0ZfluV0q6vdVjmxXqJJFV7tK6dROPRWUIrBtoLbZ9B4DcSaIs5ARJ2y38IZkn6fPu/mUz+y9Jt5nZakkPSLosgbHVr1adJAEbaRY9bm+4IVy/5ppw2dc3HqhvuIEZ01aJNvmp9zgAIPVaHq7d/SeSzqxy/FFJK1o9noZQJ4msGhwMj9MoVN9wQ/j4mmvGAzfrBgAAaFhSCxqzLQoolXWSlYvEli9n1gnpFJUhROHZbOK6gWuu4cUhgKYqlaT+fml4WFq6VOrtlTo7kx4VEJ/EFzTORWILGt1DwO7pmRhCah0H0oqFjQBaaOdOaeXK8FQzOiq1t4enoB07pO7upEcHTC+1CxpzIaqHnBxCah0H0oj+ygBaqFQKwbpUCsFaCpfR8ZGRZMcHxIVwDRTR5HUDY2NTN5cBgBj194enmmrGxsLtQB5Qcw0UEesGgNTKa03y8PD4jPVko6PS3r2tHQ/QLIRroIgqFzZO7q+8fDndQoCEVKtJ3rAhHzXJS5eG/0+1gN3eLi1Z0voxAc3AgkYAAFKgVJIWLgyXk3V2SgcOSB0drR9XXPL+/0P+saARAIAMyXtNcmdnmIHv7Awz1VK4jI4TrJEXhOuiqLWtNdtdoxYeM0BLFaEmubs7zFD39UmbNoXLAweyX/ICVCJcF8XgoLRq1cROEFHHiFWrwu1AJR4zaAQvyhoW1SRXk6ea5I4OafVq6dprwyUz1sgbwnVR9PRMbbVW2YqNBWyYjMcMGlHwF2WlkrRli7RxY7isVl9cS2/vxD2dKrW1hdsBpB8LGoukMhxFKluxAZPxmMFsTX4Rtnnz1Os5fezEsfsgOxgC6VXvgkbCddGw3TVmi8cMZquAL8ri7IQxMhIWL+7dG0pBenspnQDSgG4hmIrtrjFbPGbQiMpNiSI5DtZSvJ0+qEkGso1wXRRsd42ZTF5wVvmYec1rpEOHeMygPgV8UVaETh8A6kO4Lopa211HYSnni4xQh8kL0aLHzLJl0pe+JN1+O48ZzKygL+SL0ukD+TKXBbiojZrroojCUuV219MdR/FMDkV//dfSxReHYF35oozHDKazfXt4kTb5MRM9tgYGpEsuSXqUsWP3QWQNi2dnjwWNAGavgAvRELMCv5AnrEAKL7D6+0Op0NKlYUFqZ2fSo5qIF4ONIVwDaAzdQYCG0emj2LLyAmvLFmnduurrBNrbw/zK6tWtH1fa1Ruu57ViMAAyotZCNGaugbpEnT5QPKVSCNaVs8FReF25Ml2zwSzAbS4WNAIICroQDSg6FrXFI852jM3GAtzmYuYaQFCro4wUji9fnsuFaECRVStj2LAhfWUMWZCl2eDe3vBzrqatLdyOxjFzDSDo6QmdHCpLQKKAPTAQbgeQG5VlDFEoHB0dPz4ykuz4siZLs8GdneEFVGfn+Jjb28ePp6V8JatY0AgAQAGxqC1eWezAwQLc2WFBY5EUuPUVAKAxWSpjyIJo1rdWt5A0hlYW4DYHZSGNmrxV9EzHm2nyznrRONavD8fZSQ8AMEmWyhiyors7zFD39UmbNoXLAweoXy8awnWj0hRoe3qmdnWo7PpArSwAYJLe3okt7SuxqK1x0WzwtdeGyzTOWKO5KAtpVGWglcKir6QC7eSuDtGY2FkPAFBDmssYsrDLIVALCxrnIm1bRbOzHgBgltK2qC0ruxzOFi8Yso/tz1sl7kDb6OLEtAV9AABmKYsdN+qR1xcMRVNvuKbmei5qbRU9lxcsjdRys7MeACAHsrTLYb3oJ148hOtGNSvQNrI4sdbOetHXoVsIgGZIU9ck5EIe2wPm8QUDpke4blSzAu3kr9PWNvX7TMbOegCSkKauSUhGzC+w8tgeMI8vGDA9wnWjmhloK7t/RKarnTaTLrlk6u21jgNojqLN5NIGFDG/wMpje8A8vmDA9AjXjWpmoG1GLTeA5ivaTG4j77QhX2J+gRW1B+zsHA+k7e3jx7O4mDGPLxgwPbqFpM3kJ6bJ/bP5gwWkV1F/f2kDWmxN6FaVtvaAc0W3kHygFV9Wbd8eZrgqn5gqn7gGBsLMOIB0KlpbzKL9f1EdL7BmlLcXDEVEuM6qRvtcA0iPogSNos7UYyJeYKEg6HOdVSxOBLKtSGsmaAMK9lkApiBcA0BcihY0aANavA4xk/ECC5iCshAAiAtrJoqn6D9zShlRIJSFAECr5XEmt+gzszMpeq9vShmBKQjXABCXPAaNovXuni16fQOYhHANAKit6DOz9ZjtrroAco1wDQCojZnZmRWpQ0wtlA8BhxGuAQDTY2a2tqJ1iKmF8iHgMMI1AGB6zMzWRiu6gPIh4LB5SQ8AAJBi0+3CKDGDHXWIqWw5FwXs5cuLEyor393o6xt/fFA+hAJKrM+1mR0haUjSQ+7+WjM7RdIXJB0vaZekN7v7b6f7GvS5BoAmK3ofZ8yOe6jLj4yNEayRG1noc71W0p6K6x+VtNndl0h6XNLqREYFABiXx97daA7KhwBJCYVrMztZ0mskbSlfN0nnS9pWvsstknjGBoCk5bF3N+LHwk7gsKRqrj8h6b2SOsvXj5f0hLs/Xb6+X9LCJAYGAABmqdbCTikcX76c8iEURstnrs3stZIedvddDX7+GjMbMrOhRx55JObRAcgUeusC6UD5EHBYEmUhr5B0sZntU1jAeL6kPknHmlk0k36ypIeqfbK73+juXe7etWDBglaMF0Ba0VsXSAfKh4DDWh6u3f197n6yuy+WdLmkr7v7FZLulHRp+W5XSrq91WMDkDH01gVQVLxzl1pp2kRmo6QNZrZXoQb7poTHAyDt2JobQFHxzl1qJdbnOg70uQYgid66AIpnug2emGBoiiz0uQaAuaO3LoAi4p271CJcA8gueusCKLLKloeRuII1Nd0NI1wDyK5avXWjgE3NIYA8a+Y7d9R0N4xwDSC76K0LoKia/c4d3ZgaxoJGAACArNm+PcwgV75zVxmABwbmvitm5deLFLimu94FjYRrAACArHEPpRk9PRODbq3jc/k+dGOSRLcQAACA/GrFrph0Y2oI4RoAAAAT0Y2pYfOSHgAAAABSplY3JikcX7587jXdOUW4BgAAwERRN6bK2u0oYC9fTreQaRCuAQAAMFFUu13vcRxGzTUAAAAQE8I1AAAAEBPCNQAAABATwjUAAAAQE8I1AAAAEBPCNYB0c5e2b5+6YUGt4wAAJIhwDSDdBgelVasm7ggW7Ry2alW4HQCAlKDPNYB06+kZ33JXChsYVG7Jy0YGAIAUIVwDSLfJW+5GIbtyS14AAFLCPMP1il1dXT40NJT0MAC0grvUVlHJNjZGsAYAtIyZ7XL3rpnuR801gPSLaqwrVdZgAwCQEoRrAOkWBeuoxnpsbLwGm4ANAEgZaq4BpNvg4HiwjmqsK2uwly+XLrkk2TECAFBGuAaQbj090sBAuIxqrKOAvXw53UIAAKlCuAaQbmbVZ6ZrHQcAIEHUXAMAAAAxIVwDAAAAMSFcAwAAADEhXAMAAAAxIVwDAAAAMSFcAwAAADEhXAMAAAAxIVwDAAAAMSFcAwAAADEhXAMAAAAxIVwDAAAAMSFcAwAAADEhXAMAAAAxIVwDAAAAMSFcAwAAADEhXAMAAAAxIVwDAAAAMSFcAwAAADEhXAMAMFfu0vbt4bKe4wByi3ANAMBcDQ5Kq1ZJ69ePB2n3cH3VqnA7gEKYl/QAAADIvJ4eae1aqa8vXN+8OQTrvr5wvKcn2fEBaBnCNQAAc2UWArUUAnUUsteuDcfNkhsbgJZqeVmImc03s/80s++b2Q/N7EPl46eY2d1mttfM+s3sqFaPDQCAhlUG7AjBGiicJGqu/1fS+e5+pqRlki4ys5dJ+qikze6+RNLjklYnMDYAABoT1VhXqqzBBlAILQ/XHoyUrx5Z/ueSzpe0rXz8FkkUqAEAsiEK1lGN9djYeA02ARsolERqrs3sCEm7JC2R9DeSfizpCXd/unyX/ZIWJjE2AABmbXBwPFhHpSCVNdjLl0uXXJLsGAG0RCLh2t0PSVpmZsdK2i7pefV+rpmtkbRGkhYtWtScAQIAMBs9PdLAQLiMaqyjgL18Od1CgAJJtM+1uz8h6U5JZ0s61syisH+ypIdqfM6N7t7l7l0LFixo0UgBAJiGWZiZnrx4sdZxALmVRLeQBeUZa5nZMyRdIGmPQsi+tHy3KyXd3uqxAQAAAHORRFnIiZJuKdddt0m6zd3/xczuk/QFM/tLSd+TdFMCYwMAAAAa1vJw7e4/kPSiKsd/IumsVo8HAAAAiEuiNdcAAABAnhCuAQAAgJgQrgEAAICYEK4BAACAmBCuAQAAgJgQrgEAAICYEK4BAACAmBCuAQAAgJgQrgEAAICYEK4BAACAmBCuAQAAgJgQrgEAAICYEK4BAACAmBCuAQAAgJiYuyc9hoaZ2SOSHkh4GM+W9IuEx5BXnNvm4dw2B+e1eTi3zcO5bR7ObfMkcW5/390XzHSnTIfrNDCzIXfvSnocecS5bR7ObXNwXpuHc9s8nNvm4dw2T5rPLWUhAAAAQEwI1wAAAEBMCNdzd2PSA8gxzm3zcG6bg/PaPJzb5uHcNg/ntnlSe26puQYAAABiwsw1AAAAEBPCNQAAABATwnWdzGy+mf2nmX3fzH5oZh8qHz/FzO42s71m1m9mRyU91qwysyPM7Htm9i/l65zbGJjZPjO7x8x2m9lQ+dhxZvZVMxsuXz4r6XFmkZkda2bbzOx+M9tjZmdzbufOzJ5bfrxG/35lZus4t3NnZuvLf8PuNbOt5b9tPNfGwMzWls/rD81sXfkYj9kGmNnfm9nDZnZvxbGq59KCG8qP3x+Y2YuTG3lAuK7f/0o6393PlLRM0kVm9jJJH5W02d2XSHpc0uoEx5h1ayXtqbjOuY3Pee6+rKIn6CZJd7j7Ukl3lK9j9vokfdndnyfpTIXHL+d2jtz9R+XH6zJJfyjp15K2i3M7J2a2UNI1krrc/XRJR0i6XCoItroAAAcTSURBVDzXzpmZnS7p7ZLOUngueK2ZLRGP2UZ9VtJFk47VOpd/JGlp+d8aSZ9u0RhrIlzXyYOR8tUjy/9c0vmStpWP3yKpJ4HhZZ6ZnSzpNZK2lK+bOLfN9HqFcypxbhtiZs+UdI6kmyTJ3X/r7k+Icxu3FZJ+7O4PiHMbh3mSnmFm8yQdI+mgeK6Nw/Ml3e3uv3b3pyX9u6RV4jHbEHf/pqTHJh2udS5fL+lz5Zz2HUnHmtmJrRlpdYTrWSiXLeyW9LCkr0r6saQnyr9IkrRf0sKkxpdxn5D0Xklj5evHi3MbF5f0FTPbZWZrysdOcPeD5Y9/JumEZIaWaadIekTSzeVypi1m1i7Obdwul7S1/DHndg7c/SFJH5f0oEKo/qWkXeK5Ng73SnqlmR1vZsdIWinp98RjNk61zuVCST+tuF/ij2HC9Sy4+6Hy25QnK7z187yEh5QLZvZaSQ+7+66kx5JT3e7+YoW3zq42s3Mqb/TQj5OenLM3T9KLJX3a3V8kaVST3vLl3M5Nufb3Ykn/NPk2zu3slWtUX6/wwvAkSe2a+tY7GuDuexTKa74i6cuSdks6NOk+PGZjkvZzSbhuQPmt3zslna3w9sO88k0nS3oosYFl1yskXWxm+yR9QeEtyj5xbmNRnq2Suz+sULd6lqSfR2+blS8fTm6EmbVf0n53v7t8fZtC2ObcxuePJH3X3X9evs65nZtXSfofd3/E3Z+SNKDw/MtzbQzc/SZ3/0N3P0ehdv2/xWM2TrXO5UMK7xJEEn8ME67rZGYLzOzY8sfPkHSBwuKlOyVdWr7blZJuT2aE2eXu73P3k919scJbwF939yvEuZ0zM2s3s87oY0kXKrx9+UWFcypxbhvi7j+T9FMze2750ApJ94lzG6c3arwkROLcztWDkl5mZseU17VEj1mea2NgZr9bvlykUG/9efGYjVOtc/lFSW8pdw15maRfVpSPJIIdGutkZmcoFNAfofCi5DZ3/7CZ/YHCbOtxkr4n6f919/9NbqTZZmbnSnqPu7+Wczt35XO4vXx1nqTPu/tfmdnxkm6TtEjSA5Iuc/fJi0cwAzNbprAI9yhJP5H0VpWfH8S5nZPyi8EHJf2Bu/+yfIzH7RxZaCPbK+lphefVtynUp/JcO0dm9i2F9UJPSdrg7nfwmG2MmW2VdK6kZ0v6uaQPSBpUlXNZfqH4KYUSp19Lequ7DyUx7gjhGgAAAIgJZSEAAABATAjXAAAAQEwI1wAAAEBMCNcAAABATAjXAAAAQEwI1wDQQmZ2yMx2V/zbNPNnxfa9/97MHjaze6e5z3PN7Bvlse0xsxtbNT4AyANa8QFAC5nZiLt3JPS9z5E0Iulz7n56jfv8m6T/6+63l6+/0N3vmeP3PcLdD818TwDIPmauASBhZvZMM/tRtNujmW01s7eXP/60mQ2Z2Q/LG4BEn7PPzK4tzzAPmdmLzezfzOzHZvan1b6Pu39T0kwbWJyosLV79Dn3lL/fEWb2cTO718x+YGbvKh9fYWbfM7N7yjPjR1eM76Nm9l1Jf2xmp5rZl81sl5l9y8ye1/gZA4D0mpf0AACgYJ5hZrsrrl/r7v1m9k5JnzWzPknPcve/K9/+F+VdyI6QdIeZneHuPyjf9qC7LzOzzZI+K+kVkuYrbHH/mQbHt1nS183s25K+Iulmd39C0hpJiyUtc/enzew4M5tf/r4r3P2/zexzkv5M0ifKX+tRd3+xJJnZHZL+1N2Hzeylkv6vpPMbHCMApBbhGgBa6zfuvmzyQXf/qpn9saS/kXRmxU2XmdkahefrEyWdJikK118sX94jqcPdS5L+//bunjWKKAzD8P0ERYQgfhRaKghilcrGWiRW9hELiWBhJWrnD/AnWEiijSL+ArFSEUuJH/jRBCQRK00UxYD4WswJyrKC0VHicl/NzO6cMzPV8uzLe2Y+JFlJsrWF4jWpqtnWGjIJHAVOJZkADgGXqupLG/e2fT9fVS/b9KvAab6H6xsAScaBg8DN7k3FAGxa671J0v/AcC1J60CSMWA/8AnYBiwk2QOcAw5U1bskV+gq06tW2vbrD/urn3/7972qXgMzwExb/Di0P/sXfGzbMWBp2J8KSRo19lxL0vpwBngGTAGzSTYCW+gC6nKSncCRv30TSSbbtUmyC9gBLAK36arYG9qx7cALYHeSvW36ceDO4Dmr6j0w3yrzpDMxOE6SRoHhWpL+rc0Dj+K72BYyngTOVtU94C5woarmgIfAc+AacP9PLpzkOvAA2JdkIcn0kGGHgSdJ5oBbwPmqegNcBl4Bj9qxqar6DJyga/d4TFcx/1mv9zFgus19StdyIkkjx0fxSZIkST2xci1JkiT1xHAtSZIk9cRwLUmSJPXEcC1JkiT1xHAtSZIk9cRwLUmSJPXEcC1JkiT15Bs5QuvrC/MSGwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "positive = data[data['Admitted'].isin([1])]\n",
    "negative = data[data['Admitted'].isin([0])]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.scatter(positive['Exam 1'], positive['Exam 2'], s=50, c='b', marker='o', label='Admitted')\n",
    "ax.scatter(negative['Exam 1'], negative['Exam 2'], s=50, c='r', marker='x', label='Not Admitted')\n",
    "ax.legend()\n",
    "ax.set_xlabel('Exam 1 Score')\n",
    "ax.set_ylabel('Exam 2 Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看起来在两类间，有一个清晰的决策边界。现在我们需要实现逻辑回归，那样就可以训练一个模型来预测结果。方程实现在下面的代码示例在\"exercises\" 文件夹的 \"ex2.pdf\" 中。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# sigmoid 函数\n",
    "g 代表一个常用的逻辑函数（logistic function）为S形函数（Sigmoid function），公式为： \\\\[g\\left( z \\right)=\\frac{1}{1+{{e}^{-z}}}\\\\] \n",
    "合起来，我们得到逻辑回归模型的假设函数： \n",
    "\t\\\\[{{h}_{\\theta }}\\left( x \\right)=\\frac{1}{1+{{e}^{-{{\\theta }^{T}}X}}}\\\\] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(z):\n",
    "    return 1 / (1 + np.exp(-z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们做一个快速的检查，来确保它可以工作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAHVCAYAAADywj0dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xmc1nW9///Hi1VN3HEFFbdMW8xIWzyZO5Kh4gJY/jxlx2Nmfeu0uGQcl9LQczxt5nq0Qq8BxY0UzDRyp0RzAzNHCoU8iqICKfv798dniAEZ5oK5Zt7X8rjfbtdtruUzM0+u2zXXPHnP6/P5REoJSZIkqdF1yx1AkiRJqgYWY0mSJAmLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEgA9cn3jLbbYIu244465vr0kSZIaxGOPPfZaSqlve9tlK8Y77rgjU6ZMyfXtJUmS1CAiYkY52zlKIUmSJGExliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJKKMYR8S1EfFqRDzTxuMRET+JiOaIeCoi9q58TEmSJKlzlbNi/Atg0BoePxzYteVyCnB5x2NJkiRJXatHexuklO6PiB3XsMmRwK9SSgmYHBGbRMQ2KaWXK5RRkiRJnSmlFR878/qmm3buv6OD2i3GZdgOeKnV7Zkt91mMJUlSXsuWwfz5MG8evP02LF68+suSJW0/tqbL2nze0qVFntaXlN5935ruX5fPaX3/6spqV+nWrXgOqlglinHZIuIUinELtt9++6781pIkqVYsXVoU2XnzYO7cdbu+/OP8+Z2Xs2fPtbt07w69ehUFsVs3iFhxvfVlbe8v93MiVlyg6693q/5jPlSiGM8C+re63a/lvndJKV0FXAUwcODALv5viiRJ6lLLlsGsWdDcDC+8AC+/XF65ffvt8r5+r17Qpw9stFHxsU8f6NsXdtrp3ff36QMbbFB8zvKi2qPH2pfb1iVXdacSxXg8cHpEjAH2Bd5yvliSpAaxeDG8+GJRfpcX4OXXp0+HhQtX3n799d9dWrfdFt773nffv2qxXfV27955/s2qW+0W44hoAj4NbBERM4H/BHoCpJSuACYAg4Fm4G3gC50VVpIkZbBgQVFyW5fe5df/9reV50Y32AB22QV23x2OOKK4vvPOxcdtty1WW6UqVc5RKUa083gCvlKxRJIkqevNm7ei7K5agGfOXHlHrU02KYruRz8Kw4cX15cX4K23XjFXKtWYLt35TpIkZTRnzorCu2oBfvXVlbfdcsui7B5wwIoV3+WXzTbLk1/qZBZjSZLq2V//CtdfX1z+8peVH+vfvyi9Q4asPPKw887FDK/UYCzGkiTVmzfegJtugtGj4cEHi/s+/Wk45RTYbbei+A4YUOwIJ+mfLMaSJNWDRYtg4sSiDP/618Xt970PLrwQPvc58PwBUrssxpIk1aqU4A9/KMrw2LHw+uvFbPCXvwwnngh77+2OcNJasBhLklRrpk9fMTf8/POw3npw1FFFGT700OLEFZLWmj85kiTVgjfegBtvLFaHH3qoWAn+9KfhrLPgmGOKk19I6hCLsSRJ1WrRIpgwoSjDd9xR3N5jD7joomJuuH//3AmlumIxliSpmqQEkyevmBueM6eYGz7ttGJU4sMfdm5Y6iQWY0mSqsELL6yYG25uLg6ltnxu+JBDnBuWuoA/ZZIk5bJ8bvhXv4KHHy5Wgg84AL77XRg61LlhqYtZjCVJ6koLFxZzw9dfv2JueM894Yc/hBNOcG5YyshiLElSV5g2DX72sxVzw1ttBV/5SjEqsddezg1LVcBiLElSZ7vjDhg2rNix7uijizJ88MHODUtVxp9ISZI609VXw6mnFkeTuOMO2Hrr3IkktaFb7gCSJNWllGDkSDjlFDjsMPj97y3FUpVzxViSpEpbvLgoxL/4BXzxi3DFFdCzZ+5UktrhirEkSZU0bx589rNFKT73XLjmGkuxVCNcMZYkqVL+7//gM5+BJ58sCvHJJ+dOJGktWIwlSaqE556DQYPg1Vdh/HgYPDh3IklryWIsSVJHPfQQDBlSHH7tvvtg4MDciSStA2eMJUnqiFtvLY5JvPnm8MgjlmKphlmMJUlaVz/7GRxzTHHmuocfhp12yp1IUgdYjCVJWlvLlsEZZ8BXv1qMUNx7L2yxRe5UkjrIGWNJktbGwoXFsYlLJfjyl+GnP4Xu3XOnklQBFmNJksr11ltw9NEwaRJcdFGxahyRO5WkCrEYS5JUjpkzi0OwPfss/OpXcOKJuRNJqjCLsSRJ7XnmGTj88GLFeOLE4igUkuqOO99JkrQmv/897LcfLF0K999vKZbqmMVYkqS2jBkDhx0G224LkycXh2WTVLcsxpIkrSol+O//hhEjYN99izPbbb997lSSOpnFWJKk1pYuhW98A771LTjuOLj7bth009ypJHUBd76TJGm5BQvg85+Hm2+Gr3+9WDXu5hqS1CgsxpIkAcyZA0ceCQ8+CJdeWqwaS2ooFmNJkmbMKA7H9sILMHYsHH987kSSMrAYS5Ia2xNPFKV4wYJinnj//XMnkpSJg1OSpMb129/Cv/wL9OhRjFBYiqWGZjGWJDWmX/2qOMXzgAHFMYr33DN3IkmZWYwlSY0lJbjwQjjpJPjUp+CBB2C77XKnklQFnDGWJDWOJUvgq1+FK66AE06A666DXr1yp5JUJVwxliQ1hrffhqFDi1J8xhkwerSlWNJKXDGWJNW/lIqz2E2cCD/7GXzlK7kTSapCFmNJUv27+26YMAH+678sxZLa5CiFJKm+LVtWjE4MGFDMF0tSG1wxliTVt1IJnnyy+OhMsaQ1cMVYklS/FiyAc86BvfeGYcNyp5FU5VwxliTVr8svhxkz4JproJtrQZLWzHcJSVJ9evNN+P734dBD4eCDc6eRVAMsxpKk+nTxxTBnDowalTuJpBphMZYk1Z9Zs+BHP4LPfQ722it3Gkk1wmIsSao/554LS5cWoxSSVCaLsSSpvkybBtdeC6edBjvumDuNpBpiMZYk1Zezz4YNN4Tvfjd3Ekk1xmIsSaofDz0Et99enOluiy1yp5FUYyzGkqT6kBJ85zuwzTbw9a/nTiOpBnmCD0lSfbj9dnj4YbjqKthgg9xpJNUgV4wlSbVvyRI46yzYfXf4whdyp5FUo1wxliTVvuuugz//GW69FXr4q03SunHFWJJU2/7xD/jP/4RPfAKOPDJ3Gkk1zP9WS5Jq249/DC+/DDfdBBG500iqYa4YS5Jq12uvwahRxUrxJz+ZO42kGmcxliTVrh/8AObPhwsvzJ1EUh2wGEuSatNf/wqXXQZf/CLssUfuNJLqgMVYklSbvve94ggU556bO4mkOmExliTVnj/9CW64oTjD3Xbb5U4jqU5YjCVJtefMM2GzzYpTQEtShXi4NklSbbnnHrj7brj0Uthkk9xpJNURV4wlSbVj2TI44wzYYQc47bTcaSTVGVeMJUm1Y+xYePxxGD0aevfOnUZSnXHFWJJUGxYtgu9+Fz70ITjhhNxpJNUhV4wlSbXhiiuKYxffdRd0c11HUuWV9c4SEYMi4rmIaI6IM1fz+PYRMSki/hQRT0XE4MpHlSQ1rLlz4YIL4MAD4dBDc6eRVKfaLcYR0R24DDgc2AMYERGrnmLoHODGlNKHgeHAzysdVJLUwC65BF57DUaNgojcaSTVqXJWjPcBmlNK01NKi4AxwJGrbJOAjVqubwz8vXIRJUkN7eWXi0OzDRsGAwfmTiOpjpVTjLcDXmp1e2bLfa2dC3w+ImYCE4Cvru4LRcQpETElIqbMnj17HeJKkhrOeefB4sXwgx/kTiKpzlVq74URwC9SSv2AwcDoiHjX104pXZVSGphSGti3b98KfWtJUt167jm45ho49VTYeefcaSTVuXKK8Sygf6vb/Vrua+1k4EaAlNIjwHrAFpUIKElqYGefDeuvD+eckzuJpAZQTjF+FNg1IgZERC+KnevGr7LNi8BBABHxPopi7KyEJGndTZ4Mt9wC3/kObLll7jSSGkC7xTiltAQ4HfgN8CzF0SemRsT5ETGkZbNvAv8WEU8CTcC/ppRSZ4WWJNW5lIpCvNVW8I1v5E4jqUGUdYKPlNIEip3qWt83stX1acAnKxtNktSw7rwTHngALr8cNtwwdxpJDcJTB0mSqsvSpXDmmbDbbnDyybnTSGognhJaklRdfvlLmDoVxo2Dnj1zp5HUQFwxliRVj3fegZEjYd99YejQ3GkkNRhXjCVJ1eMnP4FZs+CGGzz1s6Qu54qxJKk6zJkDF10En/kM7L9/7jSSGpDFWJJUHS68EObOhR/+MHcSSQ3KYixJym/GDPjpT+Gkk+D978+dRlKDshhLkvIbObKYKT7//NxJJDUwi7EkKa+nnoLRo+FrX4P+/XOnkdTALMaSpLzOPBM23hjOOit3EkkNzsO1SZLymTQJJk6Eiy+GTTfNnUZSg3PFWJKUR0pwxhnF+MRXv5o7jSS5YixJymTcOHj0UbjuOlhvvdxpJMkVY0lSBosXw9lnF4dmO/HE3GkkCXDFWJKUw9VXQ3Mz3HEHdO+eO40kAa4YS5K62rx5cN55xWmfBw/OnUaS/skVY0lS17r0Unj1VRg/vjiphyRVCVeMJUld55VX4JJL4NhjYd99c6eRpJVYjCVJXeeCC2DBAvjBD3InkaR3sRhLkrrG88/DlVfCKafAbrvlTiNJ72IxliR1jZEjoXfv4qMkVSGLsSSp882ZAzffXKwWb7117jSStFoWY0lS57v55uKkHp//fO4kktQmi7EkqfOVSvDe98KHP5w7iSS1yWIsSepcs2bBfffBCSd43GJJVc1iLEnqXGPHQkowYkTuJJK0RhZjSVLnKpXgox+FXXfNnUSS1shiLEnqPM89B4895mqxpJpgMZYkdZ6mpmKueNiw3EkkqV0WY0lS50ipGKM44ADYdtvcaSSpXRZjSVLnePzx4jTQJ5yQO4kklcViLEnqHKUS9OwJQ4fmTiJJZbEYS5Iqb+lSGDMGBg+GTTfNnUaSymIxliRV3v33w9//7hiFpJpiMZYkVV5TE2y4IRxxRO4kklQ2i7EkqbIWLoRx4+Coo2CDDXKnkaSyWYwlSZX1m9/AG284RiGp5liMJUmVVSrBFlvAwQfnTiJJa8ViLEmqnPnzYfx4OP744lBtklRDLMaSpMq5/XZ45x3HKCTVJIuxJKlySiXYfnv4+MdzJ5GktWYxliRVxuzZxY53I0ZAN3+9SKo9vnNJkipj3LjijHeOUUiqURZjSVJlNDXBnnvCBz6QO4kkrROLsSSp4158ER54oBijiMidRpLWicVYktRxY8YUH0eMyJtDkjrAYixJ6rhSCT72Mdhpp9xJJGmdWYwlSR0zbRo8+aQ73UmqeRZjSVLHNDUVh2c7/vjcSSSpQyzGkqR1l1IxRnHQQbDVVrnTSFKHWIwlSevuj3+E6dMdo5BUFyzGkqR1VypB795w9NG5k0hSh1mMJUnrZulSGDsWjjgCNt44dxpJ6jCLsSRp3UyaBK+84rGLJdUNi7Ekad2USrDRRjB4cO4kklQRFmNJ0tpbsABuvhmGDoX118+dRpIqwmIsSVp7EyfC3LkejUJSXbEYS5LWXqkEW24JBxyQO4kkVYzFWJK0dubOhV//GoYNgx49cqeRpIqxGEuS1s6tt8LChY5RSKo7FmNJ0toplWDAANh339xJJKmiLMaSpPK98grce2+xWhyRO40kVZTFWJJUvptuKs5450k9JNUhi7EkqXylEnzwg7DnnrmTSFLFWYwlSeX561/hkUfc6U5S3bIYS5LKM2ZM8XH48Lw5JKmTWIwlSeUpleCTn4QddsidRJI6hcVYktS+p5+GZ55xjEJSXbMYS5LaVypB9+5w3HG5k0hSp7EYS5LWLCVoaoJDD4W+fXOnkaROU1YxjohBEfFcRDRHxJltbHN8REyLiKkRUapsTElSNo88AjNmOEYhqe71aG+DiOgOXAYcAswEHo2I8Smlaa222RU4C/hkSumNiNiyswJLkrpYqQTrrQdHHpk7iSR1qnJWjPcBmlNK01NKi4AxwKrvjv8GXJZSegMgpfRqZWNKkrJYvBhuvBGGDIE+fXKnkaROVU4x3g54qdXtmS33tbYbsFtEPBQRkyNi0Oq+UEScEhFTImLK7Nmz1y2xJKnr3HsvzJ7tGIWkhlCpne96ALsCnwZGAFdHxCarbpRSuiqlNDClNLCvO3BIUvVraoJNNoFBq13vkKS6Uk4xngX0b3W7X8t9rc0ExqeUFqeU/gr8haIoS5Jq1TvvwC23wDHHQO/eudNIUqcrpxg/CuwaEQMiohcwHBi/yja3UawWExFbUIxWTK9gTklSV7vjDpg/3zEKSQ2j3WKcUloCnA78BngWuDGlNDUizo+IIS2b/QZ4PSKmAZOAb6eUXu+s0JKkLlAqwTbbwP77504iSV2i3cO1AaSUJgATVrlvZKvrCfiPloskqda9+SZMmABf+UpxxjtJagCe+U6S9G633AKLFsGIEbmTSFKXsRhLkt6tVIJddoGBA3MnkaQuYzGWJK3s5Zfhd78rdrqLyJ1GkrqMxViStLKxYyElxygkNRyLsSRpZU1NsPfesPvuuZNIUpeyGEuSVmhuhj/+0dViSQ3JYixJWqGpqZgrHj48dxJJ6nIWY0lSIaXiaBSf+hT065c7jSR1OYuxJKnw5JPw5z97CmhJDctiLEkqlErQowccc0zuJJKUhcVYkgTLlhXzxYMGweab504jSVlYjCVJ8OCDMHOmYxSSGprFWJJUrBZvsAEMGZI7iSRlYzGWpEa3aBHceCMcdRS85z2500hSNhZjSWp0v/0tzJnjST0kNTyLsSQ1ulIJNtsMDj00dxJJyspiLEmN7B//gNtug+OOg169cqeRpKwsxpLUyH79a3j7bY9GIUlYjCWpsZVKxemf99svdxJJys5iLEmN6vXXYeJEGD4cuvnrQJJ8J5SkRnXzzbBkiWMUktTCYixJjaqpCXbfHfbaK3cSSaoKFmNJakQzZ8J99xXHLo7InUaSqoLFWJIa0dixkJIn9ZCkVizGktSISiX46Edh111zJ5GkqmExlqRG89xz8Pjj7nQnSauwGEtSo2lqKuaKhw3LnUSSqorFWJIaSUrFGMUBB8A22+ROI0lVxWIsSY3kscfg+ecdo5Ck1bAYS1IjKZWgVy8YOjR3EkmqOhZjSWoUS5cWh2kbPBg23TR3GkmqOhZjSWoU998Pf/+7xy6WpDZYjCWpUZRKsOGGcMQRuZNIUlWyGEtSI1i4EMaNg6OOgg02yJ1GkqqSxViSGsFdd8Gbb8LnPpc7iSRVLYuxJDWCpibo2xcOOih3EkmqWhZjSap38+bB+PFw3HHQs2fuNJJUtSzGklTvbr8d3nnHk3pIUjssxpJU70ol2GEH+PjHcyeRpKpmMZakejZ7Ntx9d3Hs4m6+5UvSmvguKUn1bNy44ox3ntRDktplMZakelYqwZ57wgc+kDuJJFU9i7Ek1asXX4QHHyx2uovInUaSqp7FWJLq1ZgxxUfHKCSpLBZjSapXpRJ87GMwYEDuJJJUEyzGklSPpk6FJ5/02MWStBYsxpJUj5qaisOzHX987iSSVDMsxpJUb1IqivHBB8NWW+VOI0k1w2IsSfXmj3+E6dMdo5CktWQxlqR6UypB795w9NG5k0hSTbEYS1I9WbIExo6FI46AjTbKnUaSaorFWJLqyaRJ8MorjlFI0jqwGEtSPWlqKlaKBw/OnUSSao7FWJLqxYIFcPPNMHQorLde7jSSVHMsxpJULyZMgLlzHaOQpHVkMZakelEqFcctPuCA3EkkqSZZjCWpHrz1FtxxBwwbBj165E4jSTXJYixJ9eC222DhQscoJKkDLMaSVA9KJdhpJ9hnn9xJJKlmWYwlqda98grccw+MGAERudNIUs2yGEtSrbvxRli2zDEKSeogi7Ek1bqmJvjQh2CPPXInkaSaZjGWpFo2fTo88kgxRiFJ6hCLsSTVsjFjio/Dh+fNIUl1wGIsSbUqJbjhBthvP9hhh9xpJKnmWYwlqVY9/TRMm+ZOd5JUIRZjSapVTU3QvTsce2zuJJJUFyzGklSLli0rivGhh0LfvrnTSFJdsBhLUi165BGYMcMxCkmqIIuxJNWiUgnWXx+OPDJ3EkmqG2UV44gYFBHPRURzRJy5hu2OiYgUEQMrF1GStJLFi+Gmm2DIEOjTJ3caSaob7RbjiOgOXAYcDuwBjIiId51eKSL6AP8P+EOlQ0qSWrn3Xpg925N6SFKFlbNivA/QnFKanlJaBIwBVve3uwuAUcCCCuaTJK2qVIJNNoFBg3InkaS6Uk4x3g54qdXtmS33/VNE7A30TynduaYvFBGnRMSUiJgye/bstQ4rSQ3v7bfh1luLQ7T17p07jSTVlQ7vfBcR3YBLgW+2t21K6aqU0sCU0sC+Hl5IktbeHXfA/PkejUKSOkE5xXgW0L/V7X4t9y3XB3g/8PuI+BvwMWC8O+BJUidoaoJttoFPfSp3EkmqO+UU40eBXSNiQET0AoYD45c/mFJ6K6W0RUppx5TSjsBkYEhKaUqnJJakRvXGGzBhAgwfXpzxTpJUUe0W45TSEuB04DfAs8CNKaWpEXF+RAzp7ICSpBa33AKLFjlGIUmdJFJKWb7xwIED05QpLipLUtkOOgheegmeew4icqeRpJoREY+llNod8/XMd5JUC15+GSZNKlaLLcWS1CksxpJUC8aOhZQ8qYckdSKLsSTVglIJ9t4b3vve3EkkqW5ZjCWp2j3/PDz6qDvdSVInsxhLUrVrairmiocNy51EkuqaxViSqllKxRjFpz4F/frlTiNJdc1iLEnV7IknisOzOUYhSZ3OYixJ1axUgp494ZhjcieRpLpnMZakarVsGYwZA4MGweab504jSXXPYixJ1erBB2HmTMcoJKmLWIwlqVqVSrDBBvDZz+ZOIkkNwWIsSdVo0SK46SY46ih4z3typ5GkhmAxlqRqdPfdMGeOYxSS1IUsxpJUjUqlYoe7Qw/NnUSSGobFWJKqzT/+AbffDsceWxyqTZLUJSzGklRtxo+Ht992jEKSupjFWJKqTalUnP55v/1yJ5GkhmIxlqRq8vrrcNddMGIEdPMtWpK6ku+6klRNbr4ZliwpirEkqUtZjCWpmpRKsPvusNdeuZNIUsOxGEtStXjpJbj//mKnu4jcaSSp4ViMJalajB0LKTlGIUmZWIwlqVqUSrDPPrDLLrmTSFJDshhLUjX485/hT39ytViSMrIYS1I1aGoq5oqHDcudRJIalsVYknJLqRijOPBA2Gab3GkkqWFZjCUptylToLnZU0BLUmYWY0nKrakJevWCoUNzJ5GkhmYxlqScli6FMWNg8GDYZJPcaSSpoVmMJSmn++6Dl192jEKSqoDFWJJyKpVgww3hiCNyJ5GkhmcxlqRcFi6EceOK2eL118+dRpIansVYknK56y546y1P6iFJVcJiLEm5lErQty8cdFDuJJIkLMaSlMe8eTB+PBx/PPTsmTuNJAmLsSTlcdttsGCBR6OQpCpiMZakHJqaYIcd4OMfz51EktTCYixJXW32bLj77mKnu4jcaSRJLSzGktTVbrqpOOOdYxSSVFUsxpLU1UoleP/74QMfyJ1EktSKxViSutKMGfDQQx67WJKqkMVYkrrSmDHFR4uxJFUdi7EkdaVSqTgSxYABuZNIklZhMZakrvLMM/DUU+50J0lVymIsSV2lqQm6d4fjjsudRJK0GhZjSeoKixbB9dfDQQfBVlvlTiNJWo0euQNIUkO44gp48UW48srcSSRJbXDFWJI629y5cMEFcOCBcNhhudNIktpgMZakznbJJfDaazBqlKeAlqQqZjGWpM708stw6aUwbBgMHJg7jSRpDSzGktSZzjuv2PHu+9/PnUSS1A6LsSR1lueeg2uugVNPhV12yZ1GktQOi7EkdZazz4b114fvfS93EklSGSzGktQZJk+GW26Bb38bttwydxpJUhksxpJUaSnBd75TnMjjP/4jdxpJUpk8wYckVdqdd8IDD8DPfw4bbpg7jSSpTK4YS1IlLV0KZ54Ju+4KX/pS7jSSpLXgirEkVdIvfwlTp8JNN0HPnrnTSJLWgivGklQp77wDI0fCPvvAMcfkTiNJWkuuGEtSpfzkJzBrFtxwg6d+lqQa5IqxJFXCnDlw0UXwmc/A/vvnTiNJWgcWY0mqhAsvhLlzi3IsSapJFmNJ6qgZM+CnP4WTToIPfCB3GknSOrIYS1JHjRxZzBSfd17uJJKkDrAYS1JHPPUUjB4NX/sabL997jSSpA6wGEtSR5x5Jmy8cfFRklTTPFybJK2rSZNg4kS4+GLYbLPcaSRJHeSKsSSti5TgjDOgXz84/fTcaSRJFeCKsSSti3Hj4NFH4brrYP31c6eRJFWAK8aStLYWL4azz4b3vx9OPDF3GklShZRVjCNiUEQ8FxHNEfGuPUwi4j8iYlpEPBUR90bEDpWPKklV4uqrobkZfvhD6N49dxpJUoW0W4wjojtwGXA4sAcwIiL2WGWzPwEDU0ofBMYBF1c6qCRVhXnziuMV778/DB6cO40kqYLKWTHeB2hOKU1PKS0CxgBHtt4gpTQppfR2y83JQL/KxpSkKnHppfDqqzBqVHFSD0lS3SinGG8HvNTq9syW+9pyMjBxdQ9ExCkRMSUipsyePbv8lJJUDV55BS65BI49FvbdN3caSVKFVXTnu4j4PDAQuGR1j6eUrkopDUwpDezbt28lv7Ukdb4LLoAFC+AHP8idRJLUCco5XNssoH+r2/1a7ltJRBwMfBfYP6W0sDLxJKlKPP88XHklnHIK7LZb7jSSpE5Qzorxo8CuETEgInoBw4HxrTeIiA8DVwJDUkqvVj6mJGV2zjnQuzeMHJk7iSSpk7RbjFNKS4DTgd8AzwI3ppSmRsT5ETGkZbNLgA2BmyLiiYgY38aXk6Ta8+ijcOON8M1vwtZb504jSeokkVLK8o3nFV8oAAAPqElEQVQHDhyYpkyZkuV7S1LZUoIDD4SpU+GFF6BPn9yJJElrKSIeSykNbG87TwktSWty113w+9/DT39qKZakOucpoSWpLUuXwhlnwM47FzvdSZLqmivGktSWG26Ap5+GMWOgV6/caSRJncwVY0lanQUL4Hvfg498BI47LncaSVIXcMVYklbnssvgxRfhuuugm2sIktQIfLeXpFW98UZxdrvDDiuOSCFJaggWY0la1ahR8OabxUdJUsOwGEtSay+9BD/+MXz+8/ChD+VOI0nqQhZjSWrt3HNh2TI4//zcSSRJXcxiLEnLTZ0Kv/gFnH467Lhj7jSSpC5mMZak5c46qzi73dln504iScrAYixJAA88AL/+NZx5Jmy+ee40kqQMLMaSlBJ85zuw3Xbwta/lTiNJysQTfEjSrbfC5MlwzTWwwQa500iSMnHFWFJjW7KkmC1+3/vgpJNyp5EkZeSKsaTG9r//C3/5C9x+O/TwLVGSGpkrxpIa1z/+URy3+JOfhM9+NncaSVJmLo9Ialz/8z/wf/8HN98MEbnTSJIyc8VYUmOaPRsuvhiOOgo+8YncaSRJVcBiLKkxff/7xSjFRRflTiJJqhIWY0mNZ/p0uPxyOPlk2H333GkkSVXCYiyp8ZxzTnEEinPPzZ1EklRFLMaSGsvjj0NTE3zjG7DttrnTSJKqiMVYUmM54wzYfPPiFNCSJLXi4dokNY7f/hbuuQd+9CPYeOPcaSRJVcYVY0mN4bXXivGJHXeEU0/NnUaSVIVcMZZU/6ZPh8MPhxkzilM/9+6dO5EkqQpZjCXVt8ceg8GDYfFiuPfe4vTPkiSthqMUkurXxImw//6w/vrw8MOWYknSGlmMJdWna6+Fz34WdtsNHnnEE3lIktplMZZUX1KC884rzmp30EFw332wzTa5U0mSaoAzxpLqx5Il8OUvwzXXwEknwdVXQ8+euVNJkmqEK8aS6sP8+XDkkUUpPuccuO46S7Ekaa24Yiyp9r3yChxxRHG65yuugH//99yJJEk1yGIsqbb95S/FMYpffhluu63Y4U6SpHVgMZZUuyZPLlaKI+D3v4d99smdSJJUw5wxllSbxo+HAw+ETTYpDsdmKZYkdZDFWFLtufxyOPpoeP/7ixN37LJL7kSSpDpgMZZUO1KCs8+G004r5oonTYItt8ydSpJUJ5wxllQbFi2CL30JRo+Gf/s3+PnPoYdvYZKkynHFWFL1mzsXPvOZohSffz5ceaWlWJJUcf5mkVTd/v53GDwYnnkGrr0WvvCF3IkkSXXKYiypek2bVswSv/463HknHHZY7kSSpDpmMZZUnR54AIYMgd694f77Ye+9cyeSJNU5Z4wlVZ9x4+CQQ2CrrYpjFFuKJUldwGIsqbr86Edw/PHwkY/AQw/BgAG5E0mSGoTFWFJ1WLYMvvUt+MY34Kij4J57YPPNc6eSJDUQZ4wl5bdwIfzrv8KYMXD66cWqcffuuVNJkhqMxVhSXm++WawQ33cfjBoF3/42ROROJUlqQBZjSfm89FJxOLa//AVuuAFOOCF3IklSA7MYS8rj6aeLUjxvHtx1Fxx4YO5EkqQG5853krre734H++1XXH/gAUuxJKkqWIwlda1SCQYNgv79i2MUf/CDuRNJkgRYjCV1lTfegHPPhc99Dj7xCXjwwaIcS5JUJZwxltR5Fi2CCRNg9Gi4447i9vDh8ItfFKd6liSpiliMJVVWSjB5clGGx46FOXNgyy3hy1+GE08sTu/s4dgkSVXIYiypMl54Aa6/vrg0N8N66xXHJz7xRDj0UOjh240kqbr5m0rSupszB268sVgdfvjhYiX405+Gs8+GY46BjTbKnVCSpLJZjCWtnYULV8wN33lnMTe8xx5w0UXFjnXuUCdJqlEWY0ntS6k4tNryueE33oCttoLTTitGJT78YeeGJUk1z2IsqW0vvFCU4euvL66vv/6KueFDDnFuWJJUV/ytJmllc+YUq8KjRxerxBFwwAFwzjkwdKhzw5KkumUxllTMDd9554q54cWLYc894Yc/LOaG+/XLnVCSpE5nMZYaVUrFkSRGjy6OLLF8bvj004tRib32cm5YktRQLMZSo2luXjE3PH16MTd89NFFGT74YOeGJUkNy9+AUj1KCV59tSjBzc3FjnPNzfDss/DEE8VK8IEHwsiRxdxwnz65E0uSlJ3FWKpVy5bBzJkrSm/rAvzCCzB//optu3WDHXaAXXaBUaPghBOcG5YkaRUWY6maLV4MM2asvvxOn17sNLdcr16w006w887F2ed22aW4vssuRSnu1SvbP0OSpFpgMZZyW7CgKLmtS+/y63/7GyxdumLbDTYoiu7uu8MRR6xcfvv1g+7ds/0zJEmqdRZjqTMsXAjz5sHcucXH5dffeuvdK8CzZhUzwcttvDHsuisMHAjDhxeld3kB3nprjxQhSVInKasYR8Qg4MdAd+CalNIPV3m8N/Ar4CPA68CwlNLfKhtV6kQpFSu3qxbZ1ZXbcq4vXrzm77fllkXZPeCAFcV3efndbDPLryRJGbRbjCOiO3AZcAgwE3g0IsanlKa12uxk4I2U0i4RMRwYBQzrjMCqcikVO4UtW7by9daXJUuK4rg2l3X5nLY+d/781Zfa1iMLa7LhhsXZ3/r0KS4bbVTM9i6/3vr+1d3Xv79HgZAkqQqVs2K8D9CcUpoOEBFjgCOB1sX4SODcluvjgJ9FRKTU+u/DVeCZZ+Bb31pxe9V4rW+v62PlbptS519v6/FVC2tbBXZ197d3X249ekDPnsWl9fXWlw03LIrpttu+u8S2d/097ymO8CBJkupOOcV4O+ClVrdnAvu2tU1KaUlEvAVsDrzWeqOIOAU4BWD77bdfx8gdsGQJvPnmyvet+ifr1rfX9bE1bRux4nZXXF/dfd26rbisentN93fkvtb3t1VY27qUu32PHo4gSJKkddalO9+llK4CrgIYOHBg168m77UXTJ7c5d9WkiRJ1a+cvwnPAvq3ut2v5b7VbhMRPYCNKXbCkyRJkmpCOcX4UWDXiBgQEb2A4cD4VbYZD5zUcv1Y4HdVN18sSZIkrUG7oxQtM8OnA7+hOFzbtSmlqRFxPjAlpTQe+F9gdEQ0A3MoyrMkSZJUM8qaMU4pTQAmrHLfyFbXFwDHVTaaJEmS1HU87pQkSZKExViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJAAipZTnG0fMBmZk+eawBfBapu9dD3z+Osbnr2N8/jrG569jfP46xuevY3z+1t0OKaW+7W2UrRjnFBFTUkoDc+eoVT5/HePz1zE+fx3j89cxPn8d4/PXMT5/nc9RCkmSJAmLsSRJkgQ0bjG+KneAGufz1zE+fx3j89cxPn8d4/PXMT5/HePz18kacsZYkiRJWlWjrhhLkiRJK7EYS5IkSdRpMY6I4yJiakQsi4iBqzx2VkQ0R8RzEXFYG58/ICL+0LLd2Ijo1TXJq1PLc/BEy+VvEfFEG9v9LSKebtluSlfnrFYRcW5EzGr1HA5uY7tBLa/L5og4s6tzVquIuCQi/hwRT0XErRGxSRvb+fprpb3XU0T0bvnZbm55v9ux61NWp4joHxGTImJay++S/7eabT4dEW+1+rkemSNrtWrv5zEKP2l5/T0VEXvnyFmNIuK9rV5XT0TE3Ij4+irb+PrrJD1yB+gkzwBDgStb3xkRewDDgT2BbYF7ImK3lNLSVT5/FPA/KaUxEXEFcDJweefHrk4ppWHLr0fEfwNvrWHzA1JKHnz83f4npfRfbT0YEd2By4BDgJnAoxExPqU0rasCVrHfAmellJZExCjgLOCMNrb19UfZr6eTgTdSSrtExHCK971h7/5qDWkJ8M2U0uMR0Qd4LCJ+u5qfxwdSSkdkyFcr1vTzeDiwa8tlX4rfsft2VbBqllJ6DtgL/vmzPAu4dTWb+vrrBHW5YpxSerblhbWqI4ExKaWFKaW/As3APq03iIgADgTGtdz1S+CozsxbK1qem+OBptxZ6tA+QHNKaXpKaREwhuL12vBSSnenlJa03JwM9MuZp0aU83o6kuL9DYr3u4NafsYbXkrp5ZTS4y3X5wHPAtvlTVV3jgR+lQqTgU0iYpvcoarQQcALKaVcZwpuOHVZjNdgO+ClVrdn8u43u82BN1v9Il7dNo3qX4BXUkrPt/F4Au6OiMci4pQuzFULTm/5c+G1EbHpah4v57Up+CIwsY3HfP2tUM7r6Z/btLzfvUXx/qdWWkZMPgz8YTUPfzwinoyIiRGxZ5cGq37t/Tz6nlee4bS9GOXrrxPU7ChFRNwDbL2ah76bUrq9q/PUujKfzxGsebV4v5TSrIjYEvhtRPw5pXR/pbNWozU9fxR/IryA4hfFBcB/UxQ8tSjn9RcR36X4E/cNbXyZhn39qXNExIbAzcDXU0pzV3n4cWCHlNL8lv0GbqMYC1DBn8cOatm/aQjF+NiqfP11kpotximlg9fh02YB/Vvd7tdyX2uvU/xJp0fLKsrqtqk77T2fEdGDYm77I2v4GrNaPr4aEbdS/Dm3Id4Iy309RsTVwB2reaic12bdKuP196/AEcBBqY2Drzfy6281ynk9Ld9mZsvP98YU738CIqInRSm+IaV0y6qPty7KKaUJEfHziNjCGfdCGT+PDf2eV6bDgcdTSq+s+oCvv87TaKMU44HhLXtjD6D439UfW2/Q8kt3EnBsy10nAa5Aw8HAn1NKM1f3YES8p2UnFSLiPcChFDtBNrxV5uaOZvXPy6PArlEcEaUXxZ/PxndFvmoXEYOA7wBDUkpvt7GNr7+VlfN6Gk/x/gbF+93v2vpPR6NpmbX+X+DZlNKlbWyz9fKZ7IjYh+L3qf+xoOyfx/HA/9dydIqPAW+llF7u4qjVrs2/0vr66zw1u2K8JhFxNPBToC9wZ0Q8kVI6LKU0NSJuBKZR/En2K8uPSBERE4AvpZT+TrHH+5iI+D7wJ4o3yEb3rjmniNgWuCalNBjYCri15ee0B1BKKd3V5Smr08URsRfFKMXfgH+HlZ+/liMunA78BugOXJtSmporcJX5GdCb4s+xAJNTSqf6+mtbW6+niDgfmJJSGk/xvjY6IpqBORQ/4yp8EjgReDpWHJ7ybGB7gJTSFRT/mfhyRCwB3gGG+x+Lf1rtz2NEnAr/fP4mAIMpdoJ/G/hCpqxVqeU/FIfQ8vui5b7Wz5+vv07iKaElSZIkGm+UQpIkSVoti7EkSZKExViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJAD+f5YtdEjnbx3xAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "nums = np.arange(-10, 10, step=1)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.plot(nums, sigmoid(nums), 'r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "棒极了！现在，我们需要编写代价函数来评估结果。\n",
    "代价函数：\n",
    "$J\\left( \\theta  \\right)=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[-{{y}^{(i)}}\\log \\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)-\\left( 1-{{y}^{(i)}} \\right)\\log \\left( 1-{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)]}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cost(theta, X, y):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    first = np.multiply(-y, np.log(sigmoid(X * theta.T)))\n",
    "    second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T)))\n",
    "    return np.sum(first - second) / (len(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们要做一些设置，和我们在练习1在线性回归的练习很相似。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add a ones column - this makes the matrix multiplication work out easier\n",
    "data.insert(0, 'Ones', 1)\n",
    "\n",
    "# set X (training data) and y (target variable)\n",
    "cols = data.shape[1]\n",
    "X = data.iloc[:,0:cols-1]\n",
    "y = data.iloc[:,cols-1:cols]\n",
    "\n",
    "# convert to numpy arrays and initalize the parameter array theta\n",
    "X = np.array(X.values)\n",
    "y = np.array(y.values)\n",
    "theta = np.zeros(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们来检查矩阵的维度来确保一切良好。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0.])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((100, 3), (3,), (100, 1))"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape, theta.shape, y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们计算初始化参数的代价函数(theta为0)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6931471805599453"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost(theta, X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看起来不错，接下来，我们需要一个函数来计算我们的训练数据、标签和一些参数thata的梯度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# gradient descent(梯度下降)\n",
    "* 这是批量梯度下降（batch gradient descent）  \n",
    "* 转化为向量化计算： $\\frac{1}{m} X^T( Sigmoid(X\\theta) - y )$\n",
    "$$\\frac{\\partial J\\left( \\theta  \\right)}{\\partial {{\\theta }_{j}}}=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{({{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}})x_{_{j}}^{(i)}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradient(theta, X, y):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    \n",
    "    parameters = int(theta.ravel().shape[1])\n",
    "    grad = np.zeros(parameters)\n",
    "    \n",
    "    error = sigmoid(X * theta.T) - y\n",
    "    \n",
    "    for i in range(parameters):\n",
    "        term = np.multiply(error, X[:,i])\n",
    "        grad[i] = np.sum(term) / len(X)\n",
    "    \n",
    "    return grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意，我们实际上没有在这个函数中执行梯度下降，我们仅仅在计算一个梯度步长。在练习中，一个称为“fminunc”的Octave函数是用来优化函数来计算成本和梯度参数。由于我们使用Python，我们可以用SciPy的“optimize”命名空间来做同样的事情。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们看看用我们的数据和初始参数为0的梯度下降法的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ -0.1       , -12.00921659, -11.26284221])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gradient(theta, X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在可以用SciPy's truncated newton（TNC）实现寻找最优参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-25.16131863,   0.20623159,   0.20147149]), 36, 0)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import scipy.optimize as opt\n",
    "result = opt.fmin_tnc(func=cost, x0=theta, fprime=gradient, args=(X, y))\n",
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们看看在这个结论下代价函数计算结果是什么个样子~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.20349770158947458"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost(result[0], X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们需要编写一个函数，用我们所学的参数theta来为数据集X输出预测。然后，我们可以使用这个函数来给我们的分类器的训练精度打分。\n",
    "逻辑回归模型的假设函数： \n",
    "\t\\\\[{{h}_{\\theta }}\\left( x \\right)=\\frac{1}{1+{{e}^{-{{\\theta }^{T}}X}}}\\\\] \n",
    "当${{h}_{\\theta }}$大于等于0.5时，预测 y=1\n",
    "\n",
    "当${{h}_{\\theta }}$小于0.5时，预测 y=0 。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(theta, X):\n",
    "    probability = sigmoid(X * theta.T)\n",
    "    return [1 if x >= 0.5 else 0 for x in probability]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy = 89%\n"
     ]
    }
   ],
   "source": [
    "theta_min = np.matrix(result[0])\n",
    "predictions = predict(theta_min, X)\n",
    "correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y)]\n",
    "accuracy = (sum(map(int, correct)) % len(correct))\n",
    "print ('accuracy = {0}%'.format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们的逻辑回归分类器预测正确，如果一个学生被录取或没有录取，达到89%的精确度。不坏！记住，这是训练集的准确性。我们没有保持住了设置或使用交叉验证得到的真实逼近，所以这个数字有可能高于其真实值（这个话题将在以后说明）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则化逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练的第二部分，我们将要通过加入正则项提升逻辑回归算法。如果你对正则化有点眼生，或者喜欢这一节的方程的背景，请参考在\"exercises\"文件夹中的\"ex2.pdf\"。简而言之，正则化是成本函数中的一个术语，它使算法更倾向于“更简单”的模型（在这种情况下，模型将更小的系数）。这个理论助于减少过拟合，提高模型的泛化能力。这样，我们开始吧。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设想你是工厂的生产主管，你有一些芯片在两次测试中的测试结果。对于这两次测试，你想决定是否芯片要被接受或抛弃。为了帮助你做出艰难的决定，你拥有过去芯片的测试数据集，从其中你可以构建一个逻辑回归模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和第一部分很像，从数据可视化开始吧！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Test 1</th>\n",
       "      <th>Test 2</th>\n",
       "      <th>Accepted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.051267</td>\n",
       "      <td>0.69956</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.092742</td>\n",
       "      <td>0.68494</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.213710</td>\n",
       "      <td>0.69225</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.375000</td>\n",
       "      <td>0.50219</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.513250</td>\n",
       "      <td>0.46564</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Test 1   Test 2  Accepted\n",
       "0  0.051267  0.69956         1\n",
       "1 -0.092742  0.68494         1\n",
       "2 -0.213710  0.69225         1\n",
       "3 -0.375000  0.50219         1\n",
       "4 -0.513250  0.46564         1"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path =  'ex2data2.txt'\n",
    "data2 = pd.read_csv(path, header=None, names=['Test 1', 'Test 2', 'Accepted'])\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuMAAAHjCAYAAACJlRE5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3X185HV57//3FVCRbKjcqZgF2Z7dVrTAskYqsjXFG6BbC2EFs3hHdVs8ao+77MN2l9pTTj160LVtjC2t5awoWgurnGygj66VW392RdTQglJAEhR0d0EoaDsJhQq5fn985ku+mcwkM8nM9/b1fDzymMxnJslnvvkmc81nrs91mbsLAAAAQPK60p4AAAAAUFYE4wAAAEBKCMYBAACAlBCMAwAAACkhGAcAAABSQjAOAAAApIRgHAAAAEgJwTgAAACQEoJxAAAAICUHpj2BJB1xxBF+7LHHpj0NAAAAFNjtt9/+b+5+ZDP3LVUwfuyxx2psbCztaQAAAKDAzOzBZu9LmgoAAACQEoJxAAAAICUE4wAAAEBKSpUzDgAAgODnP/+59u7dqyeffDLtqeTWQQcdpOXLl+s5z3nOor8HwTgAAEAJ7d27Vz09PTr22GNlZmlPJ3fcXY899pj27t2rFStWLPr7kKYCAABQQk8++aQOP/xwAvFFMjMdfvjhS35ngWAcAACgpAjEl6Ydx49gHAAAAEgJwTgAAAAWVKlIO3ZIW7eGy0qlPd93dHRUZqZ77723Pd9wAZ/85Cf1xBNPtPQ1X/va1/SmN72pI/MhGAcAAMC89uyRenulzZul7dvDZW9vGF+qq666SmvXrtVVV1219G/WhMUE451EMA4AAICGKhVp3bpwOTUVxqamZsYnJxf/vScnJ7Vnzx595jOf0dVXX/3s+Mc//nEdf/zxOvHEE7Vt2zZJ0sTEhN7whjfoxBNP1Jo1a3T//fdLkj7xiU/oVa96lU444QRdcsklkqQHHnhAL3vZy/S2t71Nxx13nM4991w98cQT+tSnPqX9+/frtNNO02mnnSZJuv7663XKKadozZo1Ou+88zRZfUD/+I//qJe97GVas2aNRkZGFv8gF0AwDgAAgIZ27pSmp+vfNj0dbl+sa6+9VmeeeaZ+6Zd+SYcffrhuv/12feUrX9G1116rb33rW7rzzjv1B3/wB5Kkt73tbXr/+9+vO++8U7feequOOuooXX/99RofH9e3v/1t3XHHHbr99tv19a9/XZL0/e9/X+973/t0zz336JBDDtFf/dVf6QMf+IBe8pKX6JZbbtEtt9yif/u3f9NHPvIR3Xjjjfrnf/5n9fX16c///M/15JNP6nd/93f193//97r99tv18MMPL/5BLoBgHAAAAA2Nj8+siNeampImJhb/va+66ipt2LBBkrRhwwZdddVVuvHGG/Wud71LBx98sCTpsMMOU6VS0b59+3TOOedICs12Dj74YF1//fW6/vrrddJJJ2nNmjW69957NT4+Lkk6+uijdeqpp0qS3v72t2tPnZya2267TXfffbdOPfVUrV69WldeeaUefPBB3XvvvVqxYoVWrVolM9Pb3/72xT/IBdD0BwAAAA2tWiV1d9cPyLu7pZUrF/d9H3/8cd1888363ve+JzPTM888IzPTeeed1/T3cHddfPHFes973jNr/IEHHphTdrBeGUJ31xvf+MY5+ep33HFHC49kaVgZBwAAQEODg1JXg4ixqyvcvhjXXHON3vGOd+jBBx/UAw88oB//+MdasWKFfuEXfkGf/exnn91k+fjjj6unp0fLly/X6OioJOmpp57SE088oTPOOENXXHHFs3ne+/bt0yOPPCJJ+tGPfqRvfvObkqS/+7u/09q1ayVJPT09qlRLwbz61a/WN77xDU1Ul/enpqZ033336WUve5keeOCBZ/PSO7m5lGAcAAAADfX0SLt3h8vu7jDW3T0zvmzZ4r7vVVdd9WzaSeTNb36zHnroIZ111lnq6+vT6tWr9ad/+qeSpC984Qv61Kc+pRNOOEGvec1r9PDDD+v000/XW9/6Vp1yyik6/vjjde655z4baP/yL/+yLrvsMh133HH66U9/qve+972SpAsvvFBnnnmmTjvtNB155JH63Oc+p/PPP18nnHCCTjnlFN1777066KCDdPnll+s3f/M3tWbNGr3whS9c3INsgrl7x7551vT19fnY2Fja0wAAAEjdPffco+OOO67p+09Ohs2aExMhNWVwcPGBeKc98MADetOb3qS77rqr4z+r3nE0s9vdva+ZrydnHABquUujo9LAgBTPMWw0DgAlsGyZtHFj2rMoHtJUAKDW6Ki0fr100UUhAJfC5UUXhfFqziIAIJuOPfbYRFbF24GVcQCoNTAgbdokDQ+H60NDIRAfHg7jAwPpzg8AUBgE4wBQyywE4FIIwKOgfNOmME6KCgCgTUhTAYB64gF5hEA8v9ylXbtm0o4WGgeAhBCMA0A9UY54XDyHHPnCPgAAGUUwDgC1oiAtyhGfnp7JIScgz6f4PoDod8g+AKA5HXxn6YADDtDq1av1K7/yK/qt3/ot/exnP1vwa17zmtcs6meNjo7q7rvvbvnrlnW4fiPBOADUGh2dCdKi1JShoZlgjlXU/Kn9HXZ1zf0dA6ivg+8sPf/5z9cdd9yhu+66S4cddpguu+yyBb/m1ltvXdTPWmww3mkE4wBQa2BAGhmZHaRFwdzICKuoecU+AGBxEnpn6ZRTTtG+ffuevf6JT3xCr3rVq3TCCSfokksueXY8vlLd6D6f//zndcIJJ+jEE0/UO97xDt1666267rrr9Pu///tavXq17r//ft1///0688wz9cpXvlK/9mu/pnvvvVeS9MMf/vDZjp5/9Ed/1JbHNh+qqQBALTOppkXzvOPIh0b7AAjIgfklUGHqmWee0U033aSN1a5C119/vcbHx/Xtb39b7q6zzjpLX//61/Xa17722a9pdJ/DDz9cH/nIR3TrrbfqiCOO0OOPP67DDjtMZ511lt70pjfp3HPPlSS9/vWv16c//WmtWrVK3/rWt/S+971PN998szZt2qT3vve9euc739nUSv1SsTIOACg+9gEAS9Ohd5b+8z//U6tXr9aLX/xi/eQnP9Eb3/hGSSHQvv7663XSSSdpzZo1uvfeezU+Pj7raxvd5+abb9Z5552nI444QpJ02GGHzfm5k5OTuvXWW3Xeeedp9erVes973qOHHnpIkvSNb3xD559/viTpHe94x5IeXzNYGQcAFF+jfQBSGO/v510PYD4demcpyhl/4okndMYZZ+iyyy7TBz7wAbm7Lr74Yr3nPe+ZZ0r17/MXf/EXC/7c6elpveAFL9Add9xR93ZL8N0yVsYBAMXHPgBg8RJ4Z+nggw/Wpz71Kf3Zn/2Znn76aZ1xxhm64oorNDk5KUnat2+fHnnkkVlf0+g+r3vd6/TlL39Zjz32mCTp8ccflyT19PSoUqlIkg455BCtWLFCX/7yl6sP0XXnnXdKkk499VRdffXVkqQvfvGLS35sCyEYBwAUX5TvX7va1WgcwIyEKkyddNJJOuGEE3TVVVfp9NNP11vf+tZnN1Kee+65zwbS0ap1o/u84hWv0Ic+9CH19/frxBNP1JYtWyRJGzZs0Cc+8QmddNJJuv/++/XFL35Rn/nMZ3TiiSfqFa94ha699lpJ0vDwsC677DIdf/zxszaUdop5ifLk+vr6fGxsLO1pAAAApO6ee+7Rcccdt/Ad3UPAPTAw+4Vro/EOeuyxx7RmzRo9+OCDify8ZtQ7jmZ2u7v3NfP1rIwDAACgsYy8s7R//36dcsop+uAHP5jIz0sKGzgBAACQeS95yUt03333pT2NtmNlHAAAoKTKlK7cCe04fgTjAAAAJXTQQQfpscceIyBfJHfXY489poMOOmhJ34c0FQAAgBJavny59u7dq0cffTTtqeTWQQcdpOXLly/pexCMAwAAlNBznvMcrVixIu1plB5pKgAAAEBKCMaBetylXbvmdhVrNA40i3MLABCTajBuZleY2SNmdleD283MPmVmE2b2XTNbE7vtAjMbr35ckNysUQqjo9L69bPb/EbtgNevb1u3MZQQ5xYAICbtlfHPSTpzntt/Q9Kq6seFkv5akszsMEmXSPpVSSdLusTMDu3oTFEuAwMzbX6joOmii2baAQ8MpD1DLEWaq9OcWwCAmFSDcXf/uqTH57nL2ZI+78Ftkl5gZkdJOkPSDe7+uLv/VNINmj+oB1pjJg0NzQRNXV0zwdLQUGLdxtAhaa5Oc24BAGLSXhlfSK+kH8eu762ONRqfw8wuNLMxMxujdA9aEgVNcQRLxZD26jTnFgCgKuvB+JK5++Xu3ufufUceeWTa00GeRAFaXHwlFfmV9uo05xYAoCrrwfg+SUfHri+vjjUaB9qjdqV0enruSiryLa3Vac4tAEBM1oPx6yS9s1pV5dWS/t3dH5L0VUmnm9mh1Y2bp1fHgPYYHZ27UhpfSaXiRf6ltTrNuQUAiEm1A6eZXSXp1yUdYWZ7FSqkPEeS3P3TknZLWidpQtITkt5Vve1xM/vfkr5T/VYfdvf5NoICrRkYkEZGwmW0UhoFTf39pa54UalIO3dK4+PSqlXS4KDU05P2rFpUuzo9NDRzXersCjnnFgAgxrxEb4n29fX52NhY2tMAcmvPHmndupBZMTUldXeHdOvdu6W1a9OeXQt27QpVU+Kr0/EAfWREOuectGcJAMgpM7vd3fuaui/BOIBmVCpSb2+4rNXTI+3fLy1blvy8FsU9pIPEV6fnGwcAoAWtBONZzxkHkBE7d4YV8Xqmp8PtuWEWVr5rA+5G4wAAdAjBOICmjI+H1JR6pqakiYlk5wMAQBEQjANoyqpVIUe8nu5uaeXKZOcDAEAREIwDaMrgYNisWU9XV7gdqMs9bJqt3aPUaBwASoRgHEBTenpC1ZSenpkV8u7umfHcbN5E8kZHQ/WaeB33qHrN+vXUVgdQaqnWGQeQL2vXhqopO3eGHPGVK8OKOIE45jUwMNPUSJpd133TJmqrAyg1ShsCWUTpPTQrL+dKvI57JF7nHQAKhNKGQN7xtj6alZdzJeoyGkcgDgAE40Amxd/Wj4Is3tZHPXk5V6J5xcVfQABASRGMo5yyXt0hWkWMgqyurpngitVExOXhXKl9gTA9PfcFBACUFDnjKKddu8Jb+PGAJR4wjIyEToxpc59dT3B6OhvBFbIny+dKXv7eAKBNyBkHFpKHt/Z5Wx/Nyvq5MjAQAu74Sn20oj8yko2/NwBICcE4yinrb+3ztj6alYdzxSysfNf+XTUaB+KynlYILBHBOMory9UdRkfnvjiIv3jISoUMpI9zBUWXl4pBwCIRjKO8svzWPm/ro1mcKyi6PKQVAkvABk6UU+0/89qOgFlZIQcA0DQKudPKBk6CcZQT1R0AIF+yXDEIqEE1FWAhvLUPAPmR5bRCYIkIxlFOVHcAgHzIQ8UgYAkOTHsCAAAADTWqGCSF8f5+0gqRawTjAAAgu6K0woGBuWmF/f2kFSL3CMYBAEB2RemDzY4DOUPOOAAAAJASgnEAAAAgJQTjAAAAQEoIxgEAAICUEIwDAJAU99ABuLY2dqNxAIVHMA4AbVCpSDt2SFu3hstKJe0ZIZNGR6X162c3q4ma2qxfH24HUCqUNgSAJdqzR1q3LjQGnJqSurulLVuk3bultWvTnh0yZWBgpnukFGplx7tLUjMbKB3zEr0l1tfX52NjY2lPA0CBVCpSb2/9lfCeHmn/fmnZsuTnhQyLt3ePxLtLAsg9M7vd3fuauS9pKgByJ0spITt3hhXxeqanw+3ALPF27hECcaC0CMYB5MqePWElevNmafv2cNnbG8bTMD4eUlPqmZqSJiaSnQ9yIFoZj4vnkAMoFYJxALlRqYTc7EplJgCempoZn5xMfk6rVoUc8Xq6u6WVK5OdDzIunqKyaVN4+yTKIScgB0qJYBxAbmQxJWRwUOpq8J+0qyvcDjxrdHQmEI9SU4aGZgJyqqkApUMwDiA3spgS0tMTqqb09MyskHd3z4yzeROzDAxIIyOzc8SjgHxkhGoqQAkRjAN4VpY2RtaT1ZSQtWtD1ZThYWnbtnC5f38OyhpmuQFNlue2FGbSOefM3azZaBxA4RGMA5CUvY2R9WQ5JWTZMmnjRunSS8NlLlbEs9yAJstzA4A2IhgHkMmNkfWQEtJm8QY0UdCblQY0WZ4bALQRHTgBNLUxcuPGZOfUSJQSsnNnyBFfuTKsiBOIL0K83vXw8EwTmiw0oMny3ACgjejACUBbt4bUlEa2bQvpFygo99n5P9PT2Ql2szy3vHMP6T4DA7OPaaNxAE2jAyeAlmR1YyQSkOUGNFmeWxGQlw9kAsE4gExvjEQHZbkBTZbnVhTk5QOZkGrOuJmdKWlY0gGSdrj7x2puH5J0WvXqwZJe6O4vqN72jKTvVW/7kbuflcysgeKJNkCuWxdinqmpsCLe1cXGyEJr1IBGCuP9/aHcHnMrJvLygUxILWfczA6QdJ+kN0raK+k7ks5397sb3P9/SDrJ3d9dvT7p7i2FCOSMA/ObnGRjZKlkOWc4y3MrGvLygbZrJWc8zZXxkyVNuPsPJMnMrpZ0tqS6wbik8yVdktDcgFKKamWjJKJGM82OJynLcyuSRnn5rIwDiUkzZ7xX0o9j1/dWx+Yws5dKWiHp5tjwQWY2Zma3mVnDxDYzu7B6v7FHH320HfMGACD/spSXX9SOq0AT8rKBc4Oka9z9mdjYS6vL/2+V9Ekz+2/1vtDdL3f3PnfvO/LII5OYKwAA2dcoLz8KyJOspkJlF5RYmmkq+yQdHbu+vDpWzwZJ748PuPu+6uUPzOxrkk6SdH/7pwkAQAENDEgjI7Pz76OAvL8/2Woq8couUpgDlV1QEmlu4DxQYQPn6xWC8O9Iequ7/2vN/V4m6R8lrfDqZM3sUElPuPtTZnaEpG9KOrvR5s8IGzgBAMioeNpMhMouyKlcNP1x96cl/Z6kr0q6R9KX3P1fzezDZhYvU7hB0tU++1XDcZLGzOxOSbdI+thCgTgAAMiweKnFCIE4SiDVnHF33+3uv+Tu/83dP1od+2N3vy52n//l7ttqvu5Wdz/e3U+sXn4m6bkjQ/K68Sev8waATqDjKkoqLxs4gcbyuvEnr/NOQaUi7dghbd0aLiuVtGeULRwf5F6WKrsASXP30ny88pWvdBTQ9LT7pk3uUrisdz2L8jrvhP3TP7n39Lh3d4dD090drv/TP6U9s2zg+KAQRkbm/u+L/08cGUl3fkCLJI15k/Fpahs408AGzgLL68afvM57iSqV0OlzfFxatSp0+uzpqX+/3t76K709PdL+/eXuEMrx6axmz1O0AR1XUTCtbOAkGEdx5LWlc17nvUh79kjr1oWHOTUldXeHh797t7R27ez77tghbd4c7leruzu8hilzx1COT+e0cp4CQK1cVFNBxuR9M2FeN/7kdd6LVKmEAKdSmQkgp6ZmxicnZ99/fLx+oBl93cRE++eXp9zrpI9PWbR6ngLAUhCMI8jzZsK8bvzJ67yXYOfO8DDrmZ4Ot8etWhVWJOvp7pZWrmzf3PbsCSkfmzdL27eHy97eMJ5VSR6fMmn1PAWApSAYRxDvfhYFgnnpfpalls6tyOu8l6DVldzBwdkZPHFdXeH2dsjrSmhSx6dseMcBQJIOTHsCyIh4s4Xh4ZkNhXnYTJills6tyOu8lyBayW2U41y7ktvTE3J0G+XutmtzYjMroVnMvU7q+JRNq+dp0thYChQLGzgxW8k2EyJZi63+MTkZgo+JiRAIDQ62N9DcujWkpjSybZt06aXt+3nt1unjUzZZrlLDxtKCo6pMYbSygZOVccxotJkw6yvjyI3FruQuW9bZlemsr4QupNPHp2yy+o5DPJ0qEp2z69ZRyrIQov1b8Xel42mjIyPSOeekPUu0GcE4gtoc8aGh2fWvCcjRJmvXhqAhSyu5g4PSli31byP3upyyeJ7mNZ0KLYjv35JmPxdnff8WFo1gHEGjzYRSGO/v59U42qbVldxO58hmdSUU6craOw5sLC2BPO/fwqKRM46APDVkVJI5suReI8to8lQi7N/KPTpwNkAwDuRLljfSAUnj76EJRVhYiqeNRlgZzx06cAJtlLeujEVC8xVgRpRO1dMz0+ypu3tmvPSBuJTvBnZSKZvBgZxxYF71UiS2bKGMWFLIkQVmy+LG0kzJ+wZI9m+VEsE40ABlxNKX95KDQCdkbWNppuR9A2QJm8GBNBWgIVIk0ke7d3SEu7Rr19y3/BuNI1/iAXkkD4G4FOZ4zjlz59poHIVAMA40QIpE+siRRUfkPa8Y82vUwI4XWcgo0lSABkiRyAZyZNF2ec8rRmM0sEMOUdoQaIAyYkCBUT6umHbtop08MoE64w0QjKNVSTacAZAwGqsUTxHqjKMQqDOOXMlyHe8oRWJ4WNq2LVzu308gDuQeecXp6tQmWjZAIofIGUeq8lDHmzJiQMGQV5y+aBMt6SQAwTjSQx1vAKmgsUr62EQLPItgHKlppo43K9IA2q7JxiqVSvg/ND4eqisNDobN22iDvDfnAdqIDZxIzdat0vbtjW/ftk269NLk5gNgYWUJUNm8nRA20aKg2MCJXIjqeNdDHW8ge/bsCeU+N28OL6Q3bw7X9+xJe2btFU+hi1LnpqZmxicn051fYbCJFpBEMI4U0eocyI8yBajNpNBhiWo30U5Pz+SQE5CjZAjGkRpanQP5UaYAdXy8fuddKYxPTCQ7n0JqtIk2CshHR9OeIZAYNnAiVbQ6B/KhTAFqlEJX7/GSQtcmTW6iBcqAYBypo443sqIsmxMXo0wB6uBg6HdQDyl0bRI14Wl2HCgwqqkAKJTFBtRUz5hfpRI2a9brkNvTU7y+AJwPAJailWoqBOMACmOxAVTZAs3FKluAOjlJCh2AxWklGCdNpUzcw6aYeI7efONAjiyloysNqJrT7B6PoqT7kEIHIAkE42UyOiqtXz9793q8vNTICLl6yK2lBNRl2py4VAsFqPVWz7dsKe7qOQAsFaUNy2RgYG4d13idV3avI8eWElDTgKo9ylSLHADahWC8TGrruHZ1za3zCuTUUgJqGlC1RydqkVcq0o4d0tat4bJeXj8A5BnBeNlEAXkcgTgKYCkBNQ2o2qPd6T579oSNtZs3S9u3h8ve3jAOAEVBMF42UWpKHK2HUQBLDaijzYnDw9K2beFy/37ynFvRznQfUl4AlAUbOMukNkd8aGjmusQKOXJvqR1dqZ6xNO1slkOFGwBlQTBeJqOjc3PEo5SV4eHQgphqKplUlFJxSSCgTk/0LkSjWuStpPtQ4Qa5QdlgLBHBeJkMDITyhfF/DFFA3t9PNZWMolQc8mSp705EopSXegE5FW6QKZQNxhKl2oHTzM6UNCzpAEk73P1jNbf/tqRPSNpXHfpLd99Rve0CSX9UHf+Iu1+50M+jA2fGsbowB50hUVac+8iN+VJAqVZWWq104ExtA6eZHSDpMkm/Ienlks43s5fXuetOd19d/YgC8cMkXSLpVyWdLOkSMzs0oamjU6LVhfiG0uif3Pr14faS6USpOCAPqHCTbZScjKFsMJYozTSVkyVNuPsPJMnMrpZ0tqS7m/jaMyTd4O6PV7/2BklnSrqqQ3NFEuJNiaS5qwslTKMhbxZl1q6UF7QXqXN1RAF59PwlEYijaWkG472Sfhy7vldhpbvWm83stZLuk3SRu/+4wdf21vshZnahpAsl6ZhjjmnDtNExtRtKo39qJV5dIG8WZceG3GyJl5yMRP+f1q0rcfpQo7LBJX3uQmuyXmf87yUd6+4nSLpB0oJ54bXc/XJ373P3viOPPLLtE0Sb0ZRolrQ7Q/JWNIA4UufqqM0Zn56eeZeXPh5oQprB+D5JR8euL9fMRk1Jkrs/5u5PVa/ukPTKZr8WOUVTolnSzJul+yGAWqTO1dGobHAUkJdwvxNak2aaynckrTKzFQqB9AZJb43fwcyOcveHqlfPknRP9fOvSvo/sU2bp0u6uPNTRkcl2JQoT3W708ib5a1oAPWQOlcHZYOxRGmXNlwn6ZMKpQ2vcPePmtmHJY25+3VmdqlCEP60pMclvdfd761+7bsl/WH1W33U3T+70M+jtGHG7dqVSK3WepuPoqYkpd18VGPHjrAS3ugJd3iYPF6gjCg5CTSnldKGqQbjSSMYz7gE6ozzRNKcrVtDakoj27ZJl16a3HwAZAcLGsDCWgnG6cCJ7DCrv/LdaHwRmtl8xIovb0UDaIySk0B7EYyjVNh81JzBwVA3uJ4kqrgAyDZKTgLtQzCOUmHFtzlRtZZGb0WzAgag6PK00R/5Rs44SoWc8dZMTvJWNIDyIS8eS0XOONAAK76t4a1oAGVDaVckjWAcpcPmIwBAI2z0R9IIxlFKrPgCAOphoz+S1pX2BAAAALIi2uhfDxv90QkE4wAAAFWDg2EfUT2UdkUnEIwDAABURRv9e3pmVsi7u2fG2V+EdiNnHAAAIIaN/kgSwTgApICGIkC2sdEfSSEYB4CE1WsosmULDUUAoIzIGQeABMUbikTl06amZsYnJ9OdHwAgWQTjAJCgZhqKAADKg2AcABJEQxEAQBzBOAAkiIYiAIA4gnEASBANRQAAcQTjAJAgGooAyDR3adeucNnMOJaMYBwAEhY1FBkelrZtC5f791PWEEAGjI5K69dLF100E3i7h+vr14fb0VbUGQeQirI3vaGhCIBMGhiQNm0KqwSSNDQUAvHh4TA+MJDu/ArIvERvN/T19fnY2Fja0wBKr17Tm64umt4AQCZEK+FRQC6FQHxoSDJLb145Yma3u3tfU/clGAeQpEpF6u0Nl7V6ekK6BnnTAJAy99m7zaenCcRb0EowTs44gETR9AYAMi5aGY+L55CjrQjGkYhKRdqxQ9q6NVzWWxVFOdD0BgAyLJ6ismlTWCWJcsgJyDuCDZzouHr5wVu2kB9cVlHTm3oBOU1vACBlo6MzgXiUIz40FG4bHpb6+6Vzzkl3jgVDzjg6ivxg1OKcAIAMcw8B+cDA7BzxRuOoi5xxZAb5wahF0xsAyDCzsPJdG3A3GscZ9ZiNAAAgAElEQVSSkaaCjiI/GPVETW927gznwMqVoc44gTgAoGwIxjut5G/3kB/cGUVomEPTGwAASFPpvJK3lR0cnF2mNK6rK9yO1uzZE3KuN2+Wtm8Pl729YRwAAOQLwXinxdvKRgF5idrKkh/cXpVKqExTqcy82zA1NTM+OZnu/AAAQGtIU+m02pJAUWvZErWVJT+4fZrZEEvqBwAA+dHUyriZLTez06qfP8/Mujs7rYKJB+SRdgXi7tKuXXOL8DcaT0mUH3zppeGSQHxx2BALAECxLBiMm9m7JV0naUd16KWSru3kpAqnk21lS56TXjbRhth62BALAED+NLMy/gFJr5b0H5Lk7vdJemEnJ1UonW4rW/Kc9LJhQywAAMXSTDD+pLv/V3TFzA6QVPxE53Zp1FY2CqCXunJd+/26uub+PBQGG2IBoMByknqK9jJf4BdrZn8m6SeS3iXpfZLeL2nc3S/u/PTaq6+vz8fGxpL9oUnVGXefvWQ6PU0gXmCTk2yIBYDC2bUrpJjGF9Ti73iPjIQumMg8M7vd3fuauW8z1VT+QNKFku6VtEnSVyX9zeKnVzJR+9hmxxejUU46K+OFRcMcAFlXhOZkiYunnkrheZzU08KbNxivpqR81t3fKemvk5kSWlKbIx7/w5UIyAEAiduzJ/Q+mJ4OlZ66u6UtW0I63dq1ac8uwyiHXErNpKnskXSau/88mSl1TippKp3GW1oAgAypVEJX4Epl7m09PaHvBGl1CyD1NPdaSVNpZgPn/ZL+ycwuNrMPRB9LmyLaZmAgBNzxV8zRK+uREd7SAgAkqpnmZJhHJ8shI5OaCcZ/JOkGSQdLOjL2sWRmdqaZfd/MJsxsW53bt5jZ3Wb2XTO7ycxeGrvtGTO7o/pxXTvmk0tR7nntK+ZG4wAAdBDNyZag0+WQkUkLbuB09/8pSWb2/Or1/2zHD67mo18m6Y2S9kr6jpld5+53x+72L5L63P0JM3uvpO2SokrK/+nuq9sxFwAA0B5Rc7J6ATnNyRbQqByyFMb7+0k9LaBmOnC+3My+I2lc0riZfcvMjmvDzz5Z0oS7/6Bax/xqSWfH7+Dut7j7E9Wrt0la3oafCwAAOoTmZEtA6mkpNZOmcrmkP3T35e6+XNKHJP3fNvzsXkk/jl3fWx1rZKOkr8SuH2RmY2Z2m5lxdgIAkAE0J1sCUk9LqZk64z3ufkN0xd1vrDYCSoyZvV1Sn6T+2PBL3X2fmf2ipJvN7Hvufn+dr71QoU66jjnmmETmCwBAma1dG6qm0JwMWFgzwfgDZnaxpC9Ur79d0gNt+Nn7JB0du768OjaLmb1BYTW+392fisbdfV/18gdm9jVJJylUfpnF3S9XWN1XX18fOx8AAEgAzcmA5jSTpvJuhaB5t6R/UAia392Gn/0dSavMbIWZPVfSBkmzqqKY2UkK3T7PcvdHYuOHmtnzqp8fIelUSfGNnwAAAEDmNVNN5TFJ72v3D3b3p83s9yR9VdIBkq5w9381sw9LGnP36yR9QtIySV+2kCf1I3c/S9Jxkv7GzKYVXlB8rKYKC5Aq2kADAIBmNNOB8x8lbXD3n1WvHyrpb939NxOYX1sVsgMnmpZUgFyvDXRXF22gAQAoi1Y6cDaTM/6iKBCXJHf/qZm9ZNGzA1JQL0DesqX9AXKlEn5OvA10VGt33TraQAMAgNmayRmfNrNn63ubGSVJkCvxADkKjKemZsYnJ9v3s2gDDQAAWtFMMP7Hkr5hZp81s89J+rqkP+zorIA2SjJApg00AABoRTMbOP/BzE6WdIokl/QH8comQNYlGSDTBhoAALSi4cq4mR1tZodIkrv/RNLjkl4raYOZPSeh+QFLFgXI9bQ7QKYNNAAAaMV8aSpflnSIJJnZiZJ2SXpE0smSLuv81ID2SDJApg00AABoxXxpKge7+97q529XqAP+cTPrknRn56cGtEcUCDcqN9juAJk20AAAoFnzBeMW+/x1Ci3p5e7TZkZbeeRK0gEybaAXRmMkAADmD8b/PzP7O0kPSTpc0s2SZGYvlvTzBOYGtBUBcnYkVfcdAICsmy9n/AOSdkt6WNKvuft/VcdfIul/dnpiAIopybrvAABkXcOVcXeflvS3dcb/uaMzAlBozdR95x0MABLpbCiHBeuMA0A70RgJQDNIZ0NZNNOBEwDaJsm67wDyiXQ2lAnBOIBE0RgJwEKaSWcDimK+Dpw9Zva/zeyzZvaWmtv+ovNTA1BENEYCsBDS2VAm8+WMXyHpQUn/IOndZvZmSW93959LOjWJyQEoJhojAZhPlM5WLyAnnQ1FY+71+/eY2R3uvjp2/RJJb5B0lqSb3H1NMlNsn76+Ph8bG0t7GgAAYB6VitTbGy5r9fSEF/O8eEeWmdnt7t7XzH3nyxk/yMyevd3d/0TS5yR9XdJhS5ohAABAA6SzoUzmS1P5B0mvl3RDNODunzGzhyX9ZacnBgAAyot0NpRFwzSVIiJNBQAAAJ3WrjQVAAAAAB1EMA4AAACkZMFg3Mzm5JXXGwMAAAAS5y7t2hUumxnPmGZWxr/d5BgAAACQrNFRaf166aKLZgJv93B9/fpwe4Y1XOE2sxdKOkrS883seElWvekQSQcnMDcAAABgfgMD0qZN0vBwuD40FALx4eEwPjCQ7vwWMF+6yW9Kerek5ZIu00wwXpH0Pzs8LwAAAGSZe1h1HhiQzBYe7xSzEIBLIQCPgvJNm8J4EnNYggVLG5rZW9z9SwnNp6MobQgAANAmu3aFNJB40BulhwwPSyMj0jnnJDcfd6krloE9PZ1aIN7u0oYvNLNDqt/402b2bTN7/ZJmCAAAgHyLp4dE+dpppYdEPzsunkOeYc0E4xe6+3+Y2ekKOeS/K2l7Z6cFAACATIvSQ6KAvKtrJhBPMj2k9kXA9PTcFwkZ1kwwHj2CdZI+7+53Nvl1AAAAKLJ4vnYk6Tzt0dG5LwLiLxIyXk2lmaD6TjPbLelNkr5iZss0E6ADAACgrLKQHjIwEPLT4y8CooB8ZCTz1VSaCcbfJel/STrZ3Z+QdJCkjZ2cFAAAADIuK+khZmGjaO1qfKPxjFmwk6a7P2NmvyjpjZI+Kun5Ik0FAACg3Bqlh0hhvL8/2WoqOdVMacO/lPQcSa919+PM7DBJX3X3VyUxwXaitCEAAECbZKXOeAa1u7Tha9z9PZKelCR3f1zSc5cwP2SRe6gXWvvirNE4AAAot5ynh2RFM8H4z82sS9VNm2Z2uKTpjs4KyRsdDYX74zleUS7Y+vWZ34lcBJWKtGOHtHVruKxU0p4RAADotIY542Z2oLs/LekySf9P0pFm9ieS3iLpTxKaH5ISL9wvhZyvtAr3l9CePdK6dWHvy9SU1N0tbdki7d4trV2b9uwAAECnNMwZN7N/dvc11c9fIekNkkzSje5+V3JTbB9yxhcQ3xUdSbpwfwlVKlJvb/2V8J4eaf9+admyzs9h505pfFxatUoaHAw/GwAAtK6VnPH5gvF/cfeT2jqzlBGMN8E9dNCKTE8TiHfYjh3S5s1hRbxWd3d4bbSxg8VE663Kd3WxKg8AwGK1EozPV9rwSDPb0uhGd//zlmeGbGtUuJ+V8Y4aH68fiEthfGKicz+7UgmBeHxVPprLunXJrMoDAFBm823gPEDSMkk9DT5QJFkp3F9Cq1aF1eh6urullSs797N37gy/6nqmp8PtAPKBTeBoOyqtJWK+lfGH3P3Dic0E6cpQ4f6y5S8PDobNmvV0dYXbOyXNVXkA7cMmcHREVGktHhvEF+9GRmjq0wbzBePkJZTJwED4o4oX6I8C8v7+xKqplPEJpacnPL5GedudTBOJVuUb5at3clUeQHuQboaOodJaIubbwHlYtcFP53642ZmShhVSYna4+8dqbn+epM9LeqWkxyQNuvsD1dsulrRR0jOSPuDuX13o57GBM9uyUFUkTZOT4R2BiYkQBA8OJlNFpczHHCiCtDeBo+CotLYobenAmUAgfoBCDfPfkPRySeeb2ctr7rZR0k/dfaWkIUkfr37tyyVtkPQKSWdK+qvq90OOlT1/edmy8IR56aXhMokgOFqV7+mZyVvv7p4ZJxAHso90M3RUPG01QiDeVs104OyUkyVNuPsP3P2/JF0t6eya+5wt6crq59dIer2ZWXX8and/yt1/KGmi+v2QYzyhpGPt2rACPjwsbdsWLvfvL25aEFA0aW4CRwk0qrTG5s22mS9nvNN6Jf04dn2vpF9tdB93f9rM/l3S4dXx22q+trdzU0USyF9OT7Qq32ll25wLJCHNTeAouNpKa/GccYkV8jZJc2U8EWZ2oZmNmdnYo48+mvZ0MI/Bwdn9huJ4Qsm/PXtCfvrmzdL27eGytzeMA1g80s3QMY0qrUWbOkdH055hIaS5Mr5P0tGx68urY/Xus9fMDpT0CwobOZv5WkmSu18u6XIpbOBsy8zREWlWFUFnUe0B6Kwo3SzpTeAouIxUWiu6NIPx70haZWYrFALpDZLeWnOf6yRdIOmbks6VdLO7u5ldJ+nvzOzPJb1E0ipJ305s5ugYnlCKqZnNuVR7AJYmqXSzxLiHldd4IDjfONrPrH4d8UbjWJTUgvFqDvjvSfqqQmnDK9z9X83sw5LG3P06SZ+R9AUzm5D0uELArur9viTpbklPS3q/uz+TygNB2xXuCQVszgXQOhrOoCTSXBmXu++WtLtm7I9jnz8p6bwGX/tRSR/t6AQBtAWbcwG0jIYzKImGTX+KiKY/QDpoLgRgUWg4g5xqS9MfAGgXqj0AWBQazqAEUk1TAVAebM4F0LJGDWcIyFEgrIwDmOEu7do1t7Nao/EWRZtzL700XBKIA2iotuHM9PRMDjkdIFEgBOMAZkTVC+JPdNET4vr1NHgAkBwazqAkCMaRvA6vvmIJ4tULooCc6gUA0hA1nImnpEQBedSIBigAgnEkj9XX7KpdeerqmrsyBQBJiBrL1P7faTQO5BTBOJLH6mu2Ub0AAIDEEIyXQdbSQlh9zbZG1QtIHwKA7MnaczxaRjBeBllMC2H1NZuoXgAA+ZLF53i0hGC8DLKYFsLqazZRvQAA8iWLz/FoiXmJgp++vj4fGxtLexrpyFJL4dp/FENDc6+zQp4O9xBwDwzM/h00GgcApC9Lz/GQJJnZ7e7e19R9CcZLxD3kZ0emp9P5I921K7x1Fv9HEf9HMjISdsoDAIDmZOU5HpJaC8ZJUymLLKWFUDsWAID2ydJzPFpGMF4GWduUR+1YAADaI2vP8WjZgWlPAAlotClPCuP9/aSFAACQRzzH5x4542XApjwAAIqJ5/hMYgNnA6UNxgEAAJAYNnACnUCXMwAA0GYE40Cz6HIGAADajA2cQLPiXc6kuc2KKMkIAB1XqUg7d0rj49KqVdLgoNTTk/asgMUjZxxoBV3OACA1e/ZI69aF6n1TU1J3d+hzs3u3tHZt2rMDZrCBswGCcbQFXc4AIHGVitTbGy5r9fRI+/dLy5YlPy+gHjZwAp1ClzMASMXOnWHto57p6XA7kEcE40Cz6HKGHKhUpB07pK1bw2W9VUQgj8bHQ2pKPVNT0sREsvMB2oUNnECz6HKGjKuXT7tlC/m0KIZVq8I5XS8g7+6WVq5Mfk5AO5AzDjSLLmfIMPJpUXSc48gTcsaBTjALK9+1AXejcSBB5NOi6Hp6wrs8PT1hJVwKl9E4gTjyijQVACgA8mlRBmvXhhXwnTvDOb1yZagzTiCOPCMYB4AWZLXhCPm0mE9Wz9vFWLZM2rgx2Z9ZpOOH7CFnHCgxnmBak+WGI+TTopEsn7d5wPHDYtD0pwGCcWAGTzCtyUOwy+8UtfJw3mYZxw+LxQZOAPOqVELQVqnMpDVMTc2MT06mO78sysMGySifdnhY2rYtXO7fTyBeZnk4b7OM44ckEIwDJcQTTOvyskEyyqe99NJwyapdueXlvM2qwh0/d2nXrrlN6hqNIxEE40AJFe4JJgHRBsl62CCJrOK8XZrCHb/RUWn9+tldo6Pu0uvXh9uROIJxtI5X1rnX6SeYIrZkHxwM+df1dHWF24Gs4bxdmsIdv4GB0EV6eHgmIL/oopnu0gMDac+wlAjG0TpeWedeJ59g9uwJG542b5a2bw+Xvb1hPM9oOII84rxdmsIdPzNpaGgmIO/qmgnEh4ZoXpcSqqmgdbWvpIeG5l7nDzrzOlF5owyVByYnaTiC/OG8XZrCHT/32Ssy09M8b7cZpQ0bIBhvo3hAHiEQz512P8Hs2BFWwhs1nhkeTr5ZB1CL+vooNZ6/E9FKME4HTixO9FZX/I+ZP+TcaXcnOzaGIuvqvSO0ZQu12FES872zLfE8nhJyxrE40R90XDyHHKVUuMoDKBTq66P0RkfnppTGc8jZ85UKgnG0rvaV9fT03N3ZKKXCVR5AoVBfP9+KWKUpcQMD0sjI7BXwKCAfGaGaSkpIU0HrGr2ylsJ4f790zjnpzhGpiCoMNNoYmusNT8g90qjyi/SiNjGr//zcaByJSCUYN7PDJO2UdKykByS9xd1/WnOf1ZL+WtIhkp6R9FF331m97XOS+iX9e/Xuv+3udyQxd2jmlfXAwNxX1v39vLIuuagle6EqD6AQojSqRhuMSaPKpnh6UST6Ha5bV4wqTSi3tNJUtkm6yd1XSbqper3WE5Le6e6vkHSmpE+a2Qtit/++u6+ufhCIS8k144leQddu8mg0juYUqJkSLdmRRaRR5RPpRSi6tILxsyVdWf38SklzllLd/T53H69+vl/SI5KOTGyGeUQznnzj9wd0VOEauJQE6UUourRyxl/k7g9VP39Y0ovmu7OZnSzpuZLujw1/1Mz+WNWVdXd/qsHXXijpQkk65phjljrvbIu3uZXmNuMhfSTb+P0BHUcaVf6QXoSi61jTHzO7UdKL69z0IUlXuvsLYvf9qbsf2uD7HCXpa5IucPfbYmMPKwTol0u6390/vNCcStH0h2L+mbZgsxF+f0tGQxegWMrQ2RfFk/kOnGb2fUm/7u4PRcG2u/9ynfsdohCI/x93v6bB9/p1SR909zct9HNLEYxLtLnNqKbbz/P7W7SmjzGAXOFvG3nTSjCeVs74dZIuqH5+gaRra+9gZs+VtEvS52sD8WoALzMzhXzzuzo62zyhGU8mNd1shN/fotHQBSiuKL1oeFjati1c7t9PII5iSCsY/5ikN5rZuKQ3VK/LzPrMbEf1Pm+R9FpJv21md1Q/Vldv+6KZfU/S9yQdIekjyU4/o2jGk1lNVQPg97ckVFwAio0qTSiqVDZwuvtjkl5fZ3xM0u9UP/9bSX/b4Otf19EJ5hXNeDKrqWoA/P6WhIoLAIA8ogNnkdCMJ7OaqgbA729JqLgAAMijVDZwpqU0GzixeO5hhToeEM833iSqAXQexxgAkBV52MAJZFOHGu/QbKTzOMYAgDwiTQWI62DjHZqNdB7HGACQN6SpALVovAMAAJYg801/0kIwjqbReAcAACwSOePAUtB4BwAAJIRgHIij8Q4AAEgQGziBOBrvAACABBGMA3E03gEAAAkiGAfizOqvfDcaBwAAWAJyxgEAAICUEIwDAAAAKSEYBwAAAFJCMA4AAACkhGAcQH65S7t2za3/3mgcAPKO/3uFQzAOIL9GR6X162c3ZIoaN61fH24HgCLh/17hUNoQQH4NDMx0SJVCPfh4B1XqwgMoGv7vFY55id7O6Ovr87GxsbSnAaCdohWh6IlJmt1BFQCKhv97mWdmt7t7X1P3JRgHkHvuUlcs6256mickAMXG/71MayUYJ2ccQL5FK0Rx8VxKACga/u8VCsE4gPyKv1W7aVNYGYpyKRs8MVUq0o4d0tat4bJSSWHeALBYi/i/h2xjAyeA/BodnXlCinIlh4bCbcPDUn+/dM45z959zx5p3brw3DU1JXV3S1u2SLt3S2vXpvQYGqhUpJ07pfFxadUqaXBQ6ulJe1YAUtfi/z1kHznjAPLLPTwxDQzMzpWsM16pSL299VfCe3qk/fulZcsSmvcC6r1o6OrK5osGAAlr4f8e0kPOOJBlNGxoH7OwAlT7xFNnfOfOENzWMz0dbs+CSiUE4pVKCMSlcBmNT06mO780kFoExLTwfw/5QDCO9JUtOKVhQyrGx2eC21pTU9LERLLzaSQvLxqSsmdPeEdj82Zp+/Zw2dsbxhFTtv+jQIEQjCN9ZQtO4w0bosdMw4aOW7UqpHvU090trVyZ7HwaycuLhiTwLkELyvZ/FCgQgnGkr2zBabTZJnrMXV1zN+Og7QYHZ5fkjevqCrdnQV5eNCSBdwlaULb/o0CBsIET2VDGbmI0bEhcHjZG5mmjaadt3RpSUxrZtk269NLk5pN5Zfw/CmQUGziRP/HSTJEiP4HQsCEVa9eGYHZ4OARyw8PhelYCcSkE3Lt3h8tohby7e2a8LIG4xLsELSvb/1GgIAjGkQ1lCk5p2JCqZcukjRvDiurGjdkMbvPwoiEJeUktyowy/R8FCoRgHOkrW3DaqGFD9JjZaLU0BakqkYcXDZ3GuwQtKNv/UaBAyBlH+nbtCrv948Fp/IllZKRY3cRo2NBZZTufSmByMmzWnJgIqSmDgwTic3DeA5nSSs44wTjSR3CKdqpdIRwamnud8wlFw/9RIFMIxhsgGAdKgqoSAIAUEYw3QDAOlAilIwEAKaG0IYByo6oEACAnCMYBFAtVJQAAOXJg2hMAgLZqVDpSCuP9/VSVAABkBsE4gGIZGAhl3OLVI6KAvL8/jAMAkBEE4wCKxaz+ynejcQAAUkTOOAAAAJASgnEAAAAgJakE42Z2mJndYGbj1ctDG9zvGTO7o/pxXWx8hZl9y8wmzGynmT03udkDAAAA7ZHWyvg2STe5+ypJN1Wv1/Of7r66+nFWbPzjkobcfaWkn0ra2NnpAgAAAO2XVjB+tqQrq59fKanp8gZmZpJeJ+maxXw9AKB1lYq0Y4e0dWu4rFTSnhEAFENa1VRe5O4PVT9/WNKLGtzvIDMbk/S0pI+5+6ikwyX9zN2frt5nr6TeRj/IzC6UdKEkHXPMMe2YOwCUyp490rp1oX/S1JTU3S1t2SLt3i2tXZv27AAg3zoWjJvZjZJeXOemD8WvuLubWaOWeC91931m9ouSbjaz70n691bm4e6XS7pckvr6+mi9BwAtqFRCIB5fCZ+aCpfr1kn790vLlqUzN3RepSLt3CmNj0urVkmDg1JPT9qzKgH30MAs3i9hvnHkWsfSVNz9De7+K3U+rpX0EzM7SpKql480+B77qpc/kPQ1SSdJekzSC8wseiGxXNK+Tj0OACiznTvDing909PhdhTTnj1Sb6+0ebO0fXu47O0N4+iw0VFp/XrpootCAC6Fy4suCuOjo+nOD22VVs74dZIuqH5+gaRra+9gZoea2fOqnx8h6VRJd7u7S7pF0rnzfT0AYOnGx2dWwmtNTUkTE8nOB8mIvyMS/f6npmbGJyfTnV/hDQxImzZJw8MzAflFF4XrmzbRSbhg0soZ/5ikL5nZRkkPSnqLJJlZn6T/7u6/I+k4SX9jZtMKLxo+5u53V79+q6Srzewjkv5F0meSfgAAUAarVoUc8XoBeXe3tHJl8nNCY+1KK2nmHZGNGa5jlvv0GjNpaCh8PjwcPqQQiA8NkaJSMOZenjTqvr4+HxsbS3saAJAblUpITahXPaWnh5zxLKm30bara3EbbbduDakpjWzbJl166dLm2yntPA6pcw+Tj0xPE4jnhJnd7u59zdyXDpwAgIZ6ekIQ09MTghopXEbjBOLZ0O60kugdkXqy/I5IodJrotSUuHgOOQqDYBwAMK+1a8MK+PBwWBEdHg7Xc7fKWGDt3mg7ODh7QTauqyvcnkWF2XBcmyM+PT03hxyFkVbOOAAgR5Yty3aOcNm1e6Nt9M5Ho3SPrL4jkukNx62UKxwdnQnEoxzxeA55f790zjnJPwZ0BCvjAADkXCfSSvL4jkim02taKVc4MCCNjMzerBkF5CMjVFMpGDZwAgCQgE5W+GCjbZDp41CbejI0NPc6mzMLo5UNnKSpAADQYfUqfGzZ0r4KH3lNK2m3TB8HyhWiAVbGAQDooCRXaycnw+r7xERIyRgcLE8gHpfp40C5wlKgtCEAJMVd2rVrbnWDRuNFUdbHHWnh8SdZ4SPaaHvppeEyMwFowjJ7HChXiDoIxgFgKVrZlFUkZX3ckRYef6YrfCA5lCtEA+SMA8BSDAzMPKFKczdlFbXqQVkfd6SFxx9V+KgXkKde4QPJoVwhGiBnHACWKr7iFSnDpqyyPu5Ik48/0xU+4lqpg43WcXxLpZWccYJxAGiHsm7KWuLj7mS5v0Q0+fjrVVOJKnxkpm73rl0hxSb+giL+gmNkhJVboEls4ASAJJV1U9YSH/eePWHFePNmafv2cNnbG8ZzoYXHn4sGOvHUm+hxlCn1CEiLu5fm45WvfKUDQFtNT7tv2uQuhct614toiY/7P/7Dvacn3L32o6fHvVJJ6HEsVlF/7/HHEX3k+fEAKZE05k3Gp6yMA8BSNNqUFa0wFrWqyBIfd5Ll/jqiqL/3+KbCSFn2AAApoZoKACzFwEDIpY1vvooCmv7+4r61v8THnftyf0X9vTdKvSEgBzqGlXEAWAqzsKmtNlBpNF4US3zcUbm/enJR7q+Iv3fqYAOpIBgHACRucHB2EZK4rq5wOxJW1NQbIOMIxgEAievpCWX9enpmVsi7u2fGM1F3u2yi1Jt4SkoUkEcpOQDajjrjANqHphZo0eRk2Kw5MRFSUwYHCcQB5F8rdcbZwAmgfUZHaRqClixbJm3cmPYsACA9BOMA2ifeNEQKATlNQwAAaIhgHED7xGsUDw/PBOXxlXIAAPAscsYBtJ/77FIZ09ME4gCA0mglZ5xqKgDaq1HTkBK98AcAoFkE4wDah6YhAAC0hJxxAO3TqGmIFMb7+6mmAgBADME4gPaJmobE64lHAYFbWa4AAA1USURBVHl/P9VUAACoQTAOoH3M6q98NxoHAKDkyBkHAAAAUkIwDgAAAKSEYBwAAABICcE4AAAAkBKCcQAAACAlBOMAAABASgjGAQAAgJQQjAMAAAApIRgHAAAAUkIwDgAAAKSEYBwAAABICcE4AADoPHdp165w2cw4UBIE4wAAoPNGR6X166WLLpoJvN3D9fXrw+1ACR2Y9gQAAEAJDAxImzZJw8Ph+tBQCMSHh8P4wEC68wNSksrKuJkdZmY3mNl49fLQOvc5zczuiH08aWYD1ds+Z2Y/jN22OvlHAQA5QooA0mYWAvAoIO/qmgnEh4bC7UAJpZWmsk3STe6+StJN1euzuPst7r7a3VdLep2kJyRdH7vL70e3u/sdicwaAPKKFAFkQRSQxxGIo+TSCsbPlnRl9fMrJS303tS5kr7i7k90dFYAUFTxFIEoICdFAEmLzru4+AtEoITSCsZf5O4PVT9/WNKLFrj/BklX1Yx91My+a2ZDZva8Rl9oZhea2ZiZjT366KNLmDIA5FjZUwRI00lf7QvA6em5LxCBEjLv0MlvZjdKenGdmz4k6Up3f0Hsvj919zl549XbjpL0XUkvcfefx8YelvRcSZdLut/dP7zQnPr6+nxsbKzlxwIAheEeAvHI9HTxA3EpBNzr189+8REPDkdGpHPOSXuWxcbvACViZre7e18z9+1YNRV3f0Oj28zsJ2Z2lLs/VA2sH5nnW71F0q4oEK9+72hV/Skz+6ykD7Zl0gBQZI1SBMqwMk4lj/QNDISAe2Bg5nyL3rHp7+d3gNJKK03lOkkXVD+/QNK189z3fNWkqFQDeJmZKeSb39WBOQJAcZQ9RaDsaTpZYBZWvmuPdaNxoCQ6lqYy7w81O1zSlyQdI+lBSW9x98fNrE/Sf3f336ne71hJ35B0tLtPx77+ZklHSjJJd1S/ZnKhn0uaCoDSIkUgKGuaDoBEtZKmkkownhaCcQCl5R7KF8ZTBOYbL6L4i48IK+MAOqCVYDytNBUAQJLKniJQ9jQdAJnVsQ2cAABkxujo3BzxqPnM8HDYQFiGNB0AmUMwDgAoPip5AMgognEAQPFF6TjNjgNAQsgZBwAAAFJCMA4AAACkhGAcAAAASAnBOAAAAJASgnEAAAAgJQTjAAAAQEoIxgEAAICUEIwDAAAAKSEYBwAAAFJCMA4AAACkhGAcAAAASAnBOAAAAJASgnEAAAAgJQTjAAAAQEoIxgEAAICUEIwDAAAAKSEYBwAAAFJi7p72HBJjZo9KejDteSTgCEn/lvYkcozjt3gcu8Xj2C0Nx2/xOHZLw/FbvCIfu5e6+5HN3LFUwXhZmNmYu/elPY+84vgtHsdu8Th2S8PxWzyO3dJw/BaPYxeQpgIAAACkhGAcAAAASAnBeDFdnvYEco7jt3gcu8Xj2C0Nx2/xOHZLw/FbPI6dyBkHAAAAUsPKOAAAAJASgnEAAAAgJQTjOWVmh5nZDWY2Xr08tM59TjOzO2IfT5rZQPW2z5nZD2O3rU7+UaSjmWNXvd8zseNzXWx8hZl9y8wmzGynmT03udmnr8lzb7WZfdPM/tXMvmtmg7HbSnfumdmZZvb96jmzrc7tz6ueSxPVc+vY2G0XV8e/b2ZnJDnvLGji2G0xs7ur59lNZvbS2G11/4bLpInj99tm9mjsOP1O7LYLqn/n42Z2QbIzT18Tx24odtzuM7OfxW4r9blnZleY2SNmdleD283MPlU9tt81szWx28p33rk7Hzn8kLRd0rbq59skfXyB+x8m6XFJB1evf07SuWk/jiwfO0mTDca/JGlD9fNPS3pv2o8pa8dP0i9JWlX9/CWSHpL0gur1Up17kg6QdL+kX5T0XEl3Snp5zX3eJ+nT1c83SNpZ/fzl1fs/T9KK6vc5IO3HlLFjd1rs/9p7o2NXvV73b7gsH00ev9+W9Jd1vvYwST+oXh5a/fzQtB9Tlo5dzf3/h6QrYtfLfu69VtIaSXc1uH2dpK9IMkmvlvSt6ngpzztWxvPrbElXVj+/UtLAAvc/V9JX3P2Jjs4qH1o9ds8yM5P0OknXLObrC2LB4+fu97n7ePXz/ZIekdRUJ7ICOlnShLv/wN3/S9LVCscwLn5Mr5H0+uq5drakq939KXf/oaSJ6vcriwWPnbvfEvu/dpuk5QnPMcuaOfcaOUPSDe7+uLv/VNINks7s0DyzqNVjd76kqxKZWQ64+9cVFgAbOVvS5z24TdILzOwolfS8IxjPrxe5+0PVzx+W9KIF7r9Bc/9RfLT69tCQmT2v7TPMrmaP3UFmNmZmt0XpPZIOl/Qzd3+6en2vpN4OzjWLWjr3zOxkhZWl+2PDZTr3eiX9OHa93jnz7H2q59a/K5xrzXxtkbX6+DcqrLZF6v0Nl0mzx+/N1b/Ha8zs6Ba/tqiafvzV1KgVkm6ODZf93FtIo+NbyvPuwLQngMbM7EZJL65z04fiV9zdzaxhjcrqq83jJX01NnyxQiD1XIU6n1slfXipc86KNh27l7r7PjP7RUk3m9n3FIKkwmvzufcFSRe4+3R1uNDnHtJhZm+X1CepPzY852/Y3e+v/x1K6+8lXeXuT5nZexTeoXldynPKmw2SrnH3Z2JjnHtoGsF4hrn7GxrdZmY/MbOj3P2hasDzyDzf6i2Sdrn7z2PfO1rZfMrMPivpg22ZdEa049i5+77q5Q/M7GuSTpL0/xTeTjuwuoK5XNK+tj+AlLXj+JnZIZL+QdKHqm9DRt+70OdeHfskHR27Xu+cie6z18wOlPQLkh5r8muLrKnHb2ZvUHih2O/uT0XjDf6GyxQQLXj83P2x2NUdCntCoq/99Zqv/VrbZ5hdrfztbZD0/vgA596CGh3fUp53pKnk13WSol3GF0i6dp77zsllqwZRUQ70gKS6O54LasFjZ2aHRukTZnaEpFMl3e1hh8ktCjn4Db++4Jo5fs+VtEshJ/CamtvKdu59R9IqC1V4nqvwxF1bXSF+TM+VdHP1XLtO0gYL1VZWSFol6dsJzTsLFjx2ZnaSpL+RdJa7PxIbr/s3nNjMs6GZ43dU7OpZku6pfv5VSadXj+Ohkk7X7HdXi66Zv1vZ/9/e/Yb8NcZxHH9/Rv6EMJQ9WZq0KbQYaSGk1VrRkj8hbEqz5rEHssYUD5RSa2E2DzwgjyjK2qwoQ9LNzQqTNv+2Jfm3/Bn7enCu6ddsw6/l7Lf7/arT/fudc65znXN13e27677O9U1m0L1ouHFgn33vn70I3NpWVbkY+L4N1EzMftf3G6Ruw21080nXA58A64DJbf8sYNXAeWfQ/U9z0l7lXwXG6QKhZ4Dj+36mQ6ntgNmtfd5rP+8YKD+NLiDaDDwPHN33Mx2C7XcLsAsYG9hmTtS+R7dywMd0I2P3tn0P0AWQAMe0vrS59a1pA2XvbeU+Aub2/SyHYNutA7YP9LMX2/79/g5PpO1ftN9DwIetnTYAMwbKLmx9cjOwoO9nOdTarn1fBjy8V7kJ3/foBgC/bv8OfEH3PsciYFE7HmBFa9txYNZA2QnX79IeXJIkSdL/zGkqkiRJUk8MxiVJkqSeGIxLkiRJPTEYlyRJknpiMC5JkiT1xGBckkZAklOSjLVtW5IvB74f9R+uszDJvrKrkuSGJJuS7E4ycz/nHJFkRZIPkownebulA5ckDcEMnJI0AqrLlDgTIMky4KeqemSISy0E3gW27ePYOF0iptUHKH8T3Vrz51XV7iRTgR+GuI+/DGS0laQJx2BckkZcktvo0nEfBbwBLKH7y+caugA+wBN0yXFmAs8l+Rm4qKp+23OdqtrUrneg6qYAX1fV7lZm68B9zAOWA0cA26tqTstAuJouAdlPwJ1V9UGSB4GpwJnAZ0lup0vFfgldEqTHqmrV8K0iSaPBYFySRliSc4D5wOyq+j3JE3Spuz8FTq2qc9t5J1XVd0nuBpZU1diQVT4LvJ7kcrpMrM9U1Vib+rISuLSqtiSZ3M5fDrxVVVcnmQM8TZetFWAGcFlV/ZJkMbCjqi5qqcTfTLJ2MNiXpMORwbgkjbargAuBd9qI9rHA58ArwPQkjwEvAWsPRmVVtTXJdODKtm1IMh84GdhQVVvaed+2IpcA89q+tUmeTnJcO/ZCVf3SPs8Bzk5yY/t+InAWYDAu6bBmMC5Joy3A6qq6728HkvOAuXRTWK4F7jwYFbYA+mXg5STfANcArw1xqZ0DnwMsrqr1B+EWJWlkuJqKJI22dcD1bW72nlVXpiY5DUhVPQ8sBc5v5/8InDBsZUkuSDKlfZ4EnAtsoZurfsWelVUGpqm8Dtzc9l0FfFlVO/924W4kf3GSI9u505McO+x9StKocGRckkZYVY0nuR9Y14LjXcAi4A/gqXRzVwq4pxVZA6za1wucSa4DHgVOA15J8k5VzdurytOBJ9tyigE2Aiur6tckdwEvtDq/ohuVXwqsTvI+3QucC/bzKI/TvdA51qbb7KAbcZekw1qqqu97kCRJkiYkp6lIkiRJPTEYlyRJknpiMC5JkiT1xGBckiRJ6onBuCRJktQTg3FJkiSpJwbjkiRJUk/+BODlUaUbUGyUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "positive = data2[data2['Accepted'].isin([1])]\n",
    "negative = data2[data2['Accepted'].isin([0])]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.scatter(positive['Test 1'], positive['Test 2'], s=50, c='b', marker='o', label='Accepted')\n",
    "ax.scatter(negative['Test 1'], negative['Test 2'], s=50, c='r', marker='x', label='Rejected')\n",
    "ax.legend()\n",
    "ax.set_xlabel('Test 1 Score')\n",
    "ax.set_ylabel('Test 2 Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "哇，这个数据看起来可比前一次的复杂得多。特别地，你会注意到其中没有线性决策界限，来良好的分开两类数据。一个方法是用像逻辑回归这样的线性技术来构造从原始特征的多项式中得到的特征。让我们通过创建一组多项式特征入手吧。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Accepted</th>\n",
       "      <th>Ones</th>\n",
       "      <th>F10</th>\n",
       "      <th>F20</th>\n",
       "      <th>F21</th>\n",
       "      <th>F30</th>\n",
       "      <th>F31</th>\n",
       "      <th>F32</th>\n",
       "      <th>F40</th>\n",
       "      <th>F41</th>\n",
       "      <th>F42</th>\n",
       "      <th>F43</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.051267</td>\n",
       "      <td>0.002628</td>\n",
       "      <td>0.035864</td>\n",
       "      <td>0.000135</td>\n",
       "      <td>0.001839</td>\n",
       "      <td>0.025089</td>\n",
       "      <td>0.000007</td>\n",
       "      <td>0.000094</td>\n",
       "      <td>0.001286</td>\n",
       "      <td>0.017551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.092742</td>\n",
       "      <td>0.008601</td>\n",
       "      <td>-0.063523</td>\n",
       "      <td>-0.000798</td>\n",
       "      <td>0.005891</td>\n",
       "      <td>-0.043509</td>\n",
       "      <td>0.000074</td>\n",
       "      <td>-0.000546</td>\n",
       "      <td>0.004035</td>\n",
       "      <td>-0.029801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.213710</td>\n",
       "      <td>0.045672</td>\n",
       "      <td>-0.147941</td>\n",
       "      <td>-0.009761</td>\n",
       "      <td>0.031616</td>\n",
       "      <td>-0.102412</td>\n",
       "      <td>0.002086</td>\n",
       "      <td>-0.006757</td>\n",
       "      <td>0.021886</td>\n",
       "      <td>-0.070895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.375000</td>\n",
       "      <td>0.140625</td>\n",
       "      <td>-0.188321</td>\n",
       "      <td>-0.052734</td>\n",
       "      <td>0.070620</td>\n",
       "      <td>-0.094573</td>\n",
       "      <td>0.019775</td>\n",
       "      <td>-0.026483</td>\n",
       "      <td>0.035465</td>\n",
       "      <td>-0.047494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.513250</td>\n",
       "      <td>0.263426</td>\n",
       "      <td>-0.238990</td>\n",
       "      <td>-0.135203</td>\n",
       "      <td>0.122661</td>\n",
       "      <td>-0.111283</td>\n",
       "      <td>0.069393</td>\n",
       "      <td>-0.062956</td>\n",
       "      <td>0.057116</td>\n",
       "      <td>-0.051818</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Accepted  Ones       F10       F20       F21       F30       F31       F32  \\\n",
       "0         1     1  0.051267  0.002628  0.035864  0.000135  0.001839  0.025089   \n",
       "1         1     1 -0.092742  0.008601 -0.063523 -0.000798  0.005891 -0.043509   \n",
       "2         1     1 -0.213710  0.045672 -0.147941 -0.009761  0.031616 -0.102412   \n",
       "3         1     1 -0.375000  0.140625 -0.188321 -0.052734  0.070620 -0.094573   \n",
       "4         1     1 -0.513250  0.263426 -0.238990 -0.135203  0.122661 -0.111283   \n",
       "\n",
       "        F40       F41       F42       F43  \n",
       "0  0.000007  0.000094  0.001286  0.017551  \n",
       "1  0.000074 -0.000546  0.004035 -0.029801  \n",
       "2  0.002086 -0.006757  0.021886 -0.070895  \n",
       "3  0.019775 -0.026483  0.035465 -0.047494  \n",
       "4  0.069393 -0.062956  0.057116 -0.051818  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "degree = 5\n",
    "x1 = data2['Test 1']\n",
    "x2 = data2['Test 2']\n",
    "\n",
    "data2.insert(3, 'Ones', 1)\n",
    "\n",
    "for i in range(1, degree):\n",
    "    for j in range(0, i):\n",
    "        data2['F' + str(i) + str(j)] = np.power(x1, i-j) * np.power(x2, j)\n",
    "\n",
    "data2.drop('Test 1', axis=1, inplace=True)\n",
    "data2.drop('Test 2', axis=1, inplace=True)\n",
    "\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们需要修改第1部分的成本和梯度函数，包括正则化项。首先是成本函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# regularized cost（正则化代价函数）\n",
    "$$J\\left( \\theta  \\right)=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[-{{y}^{(i)}}\\log \\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)-\\left( 1-{{y}^{(i)}} \\right)\\log \\left( 1-{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)]}+\\frac{\\lambda }{2m}\\sum\\limits_{j=1}^{n}{\\theta _{j}^{2}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def costReg(theta, X, y, learningRate):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    first = np.multiply(-y, np.log(sigmoid(X * theta.T)))\n",
    "    second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T)))\n",
    "    reg = (learningRate / (2 * len(X))) * np.sum(np.power(theta[:,1:theta.shape[1]], 2))\n",
    "    return np.sum(first - second) / len(X) + reg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请注意等式中的\"reg\" 项。还注意到另外的一个“学习率”参数。这是一种超参数，用来控制正则化项。现在我们需要添加正则化梯度函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们要使用梯度下降法令这个代价函数最小化，因为我们未对${{\\theta }_{0}}$ 进行正则化，所以梯度下降算法将分两种情形：\n",
    "\\begin{align}\n",
    "  & Repeat\\text{ }until\\text{ }convergence\\text{ }\\!\\!\\{\\!\\!\\text{ } \\\\ \n",
    " & \\text{     }{{\\theta }_{0}}:={{\\theta }_{0}}-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}}]x_{_{0}}^{(i)}} \\\\ \n",
    " & \\text{     }{{\\theta }_{j}}:={{\\theta }_{j}}-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}}]x_{j}^{(i)}}+\\frac{\\lambda }{m}{{\\theta }_{j}} \\\\ \n",
    " & \\text{          }\\!\\!\\}\\!\\!\\text{ } \\\\ \n",
    " & Repeat \\\\ \n",
    "\\end{align}\n",
    "\n",
    "对上面的算法中 j=1,2,...,n 时的更新式子进行调整可得： \n",
    "${{\\theta }_{j}}:={{\\theta }_{j}}(1-a\\frac{\\lambda }{m})-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{({{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}})x_{j}^{(i)}}$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradientReg(theta, X, y, learningRate):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    \n",
    "    parameters = int(theta.ravel().shape[1])\n",
    "    grad = np.zeros(parameters)\n",
    "    \n",
    "    error = sigmoid(X * theta.T) - y\n",
    "    \n",
    "    for i in range(parameters):\n",
    "        term = np.multiply(error, X[:,i])\n",
    "        \n",
    "        if (i == 0):\n",
    "            grad[i] = np.sum(term) / len(X)\n",
    "        else:\n",
    "            grad[i] = (np.sum(term) / len(X)) + ((learningRate / len(X)) * theta[:,i])\n",
    "    \n",
    "    return grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "就像在第一部分中做的一样，初始化变量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set X and y (remember from above that we moved the label to column 0)\n",
    "cols = data2.shape[1]\n",
    "X2 = data2.iloc[:,1:cols]\n",
    "y2 = data2.iloc[:,0:1]\n",
    "\n",
    "# convert to numpy arrays and initalize the parameter array theta\n",
    "X2 = np.array(X2.values)\n",
    "y2 = np.array(y2.values)\n",
    "theta2 = np.zeros(11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们初始学习率到一个合理值。，果有必要的话（即如果惩罚太强或不够强）,我们可以之后再折腾这个。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "learningRate = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，让我们尝试调用新的默认为0的theta的正则化函数，以确保计算工作正常。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6931471805599454"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costReg(theta2, X2, y2, learningRate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.00847458, 0.01878809, 0.05034464, 0.01150133, 0.01835599,\n",
       "       0.00732393, 0.00819244, 0.03934862, 0.00223924, 0.01286005,\n",
       "       0.00309594])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gradientReg(theta2, X2, y2, learningRate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们可以使用和第一部分相同的优化函数来计算优化后的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 0.53010249,  0.29075567, -1.60725763, -0.5821382 ,  0.01781027,\n",
       "        -0.21329509, -0.40024142, -1.37144139,  0.02264303, -0.9503358 ,\n",
       "         0.0344085 ]), 22, 1)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result2 = opt.fmin_tnc(func=costReg, x0=theta2, fprime=gradientReg, args=(X2, y2, learningRate))\n",
    "result2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，我们可以使用第1部分中的预测函数来查看我们的方案在训练数据上的准确度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy = 0.6610169491525424%\n"
     ]
    }
   ],
   "source": [
    "theta_min = np.matrix(result2[0])\n",
    "predictions = predict(theta_min, X2)\n",
    "correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y2)]\n",
    "accuracy = (sum(map(int, correct)) / len(correct))\n",
    "print ('accuracy = {0}%'.format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然我们实现了这些算法，值得注意的是，我们还可以使用高级Python库像scikit-learn来解决这个问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/wangboyuan/anaconda3/envs/py36_ml/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='warn',\n",
       "          n_jobs=None, penalty='l2', random_state=None, solver='warn',\n",
       "          tol=0.0001, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import linear_model#调用sklearn的线性回归包\n",
    "model = linear_model.LogisticRegression(penalty='l2', C=1.0)\n",
    "model.fit(X2, y2.ravel())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6610169491525424"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.score(X2, y2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这就是练习2的全部！ 敬请期待下一个练习：多类图像分类。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
